michael g michael g - 3 months ago 28
Scala Question

In akka-http, do response entities need to be consumed in a sink when the response is anything other than 200?

On this akka-http documentation page there is a warning at the bottom:


Be sure to consume the response entities
dataBytes:Source[ByteString,Unit]
by for example connecting it to a
Sink (for example
response.entity.dataBytes.runWith(Sink.ignore)
if
you don't care about the response entity), since otherwise Akka HTTP
(and the underlying Streams infrastructure) will understand the lack
of entity consumption as a back-pressure signal and stop reading from
the underlying TCP connection!

This is a feature of Akka HTTP that allows consuming entities (and
pulling them through the network) in a streaming fashion, and only on
demand when the client is ready to consume the bytes - it may be a bit
suprising at first though.


Why is this not needed in the case where the response's Status Code is not
StatusCodes.OK
? Or is it in fact needed and the code example shown on that page (also below) missing that?

def receive = {
case HttpResponse(StatusCodes.OK, headers, entity, _) =>
log.info("Got response, body: " + entity.dataBytes.runFold(ByteString(""))(_ ++ _))
case HttpResponse(code, _, _, _) =>
//why not here?
log.info("Request failed, response code: " + code)
}

Answer

The response entity should always be consumed however in practice the error responses are typically HttpEntity.Strict types and a failure to consume those won't cause any issues with back pressure.

For the interested, as of Akka 2.4.9 this behaviour is from SlotProcessor.running where it wraps the HttpEntity using HttpEntity.captureTermination. This wrapping is used to signal that the response has been consumed and thus providing the right backpressure signals. For a HttpEntity.Strict since the entity body is already in memory the captureTermination method will return a Future.success(()) whereas the other types will wrap the Source and return a Future that completes when the underlying Source is completed.

Regarding the HttpEntity type returned, there aren't any guarantees, as far as the API is concerned, that certain responses will be HttpEntity.Strict which is why the recommendation is to always consume the response. HttpResponseParser is doing the work of parsing the responses. As I understand when reading the response if the body is already read into memory (based on buffer size) and there is no transfer encoding then a HttpEntity.Strict can be returned as we've already done all the work. Otherwise a HttpEntity.Default or HttpEntity.Chunked will be returned.

On last catch, if you use the HttpEntity as part of a server response (e.g. in a complete directive) then its possible for the server to timeout and not bother consuming the entity. You can use the withRequestTimeoutResponse directive to consume response in this case.

Comments