perry perry - 1 year ago 50
Scala Question

Unable to save file to s3 using scala akka-http

Trying to save a file to s3. File is coming in as a

Source[ByteString, Any]
so having to convert it to an
InputStream
for the
PutObjectRequest
. Then calculating the
ObjectMetadata
content length and md5. Here's my code followed by the error message. Any input would be appreciated. Thanks.

def uploadFile(source: Source[ByteString, _]) = {

val stream: InputStream = source.runWith(StreamConverters.asInputStream(FiniteDuration(3, TimeUnit.SECONDS)))

val metadata = new ObjectMetadata()
metadata.setContentLength(IOUtils.toByteArray(stream).length)
metadata.setContentMD5(Base64.encodeBase64(DigestUtils.md5(stream)).toString)

s3.putObject(new PutObjectRequest("bucketName", "key", stream, metadata))
}


Stack trace error message:
com.amazonaws.services.s3.model.AmazonS3Exception with message The Content-MD5 you specified was invalid.

Answer Source

A non-blocking, more reactive alternative would be to use Alpakka S3 sink.

It would solve all your encoding and S3 protocol issues for you, plus it will be fully backpressure-enabled for an enhanced scalability.

Furthermore, it comes in the form of Sink[ByteString,_] so it doesn't need further data adaptation/conversion in your case.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download