Robin M Robin M - 3 months ago 21
Groovy Question

Base64Encode for AWS not matching Amazon Example

I seem to be failing at the first hurdle in my quest for Amazon S3 Browser-Based Uploads.

Using Amazon's example shown here : http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html

Using Groovy/Java, I'm unable to encode the example post policy and produce the same encoding as Amazon.

{ "expiration": "2015-12-30T12:00:00.000Z",
"conditions": [
{"bucket": "sigv4examplebucket"},
["starts-with", "$key", "user/user1/"],
{"acl": "public-read"},
{"success_action_redirect": "http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html"},
["starts-with", "$Content-Type", "image/"],
{"x-amz-meta-uuid": "14365123651274"},
{"x-amz-server-side-encryption": "AES256"},
["starts-with", "$x-amz-meta-tag", ""],

{"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
{"x-amz-algorithm": "AWS4-HMAC-SHA256"},
{"x-amz-date": "20151229T000000Z" }
]
}


I have this defined as a Groovy multi line string:

String policy_document = '''
{ "expiration": "2015-12-30T12:00:00.000Z",
"conditions": [
{"bucket": "sigv4examplebucket"},
["starts-with", "$key", "user/user1/"],
{"acl": "public-read"},
{"success_action_redirect": "http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html"},
["starts-with", "$Content-Type", "image/"],
{"x-amz-meta-uuid": "14365123651274"},
{"x-amz-server-side-encryption": "AES256"},
["starts-with", "$x-amz-meta-tag", ""],

{"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
{"x-amz-algorithm": "AWS4-HMAC-SHA256"},
{"x-amz-date": "20151229T000000Z" }
]
}
'''


I'm then encoding and verifying with an assert which fails.

String base64Policy = (new BASE64Encoder()).encode(policy_document.replaceAll("\n", "").replaceAll("\r", "").getBytes("UTF-8"))
assert 'eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLA0KICAiY29uZGl0aW9ucyI6IFsNCiAgICB7ImJ1Y2tldCI6ICJzaWd2NGV4YW1wbGVidWNrZXQifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwNCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LA0KICAgIHsic3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiOiAiaHR0cDovL3NpZ3Y0ZXhhbXBsZWJ1Y2tldC5zMy5hbWF6b25hd3MuY29tL3N1Y2Nlc3NmdWxfdXBsb2FkLmh0bWwifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sDQogICAgeyJ4LWFtei1tZXRhLXV1aWQiOiAiMTQzNjUxMjM2NTEyNzQifSwNCiAgICB7IngtYW16LXNlcnZlci1zaWRlLWVuY3J5cHRpb24iOiAiQUVTMjU2In0sDQogICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sDQoNCiAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlPU0ZPRE5ON0VYQU1QTEUvMjAxNTEyMjkvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LA0KICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwNCiAgICB7IngtYW16LWRhdGUiOiAiMjAxNTEyMjlUMDAwMDAwWiIgfQ0KICBdDQp9' == base64Policy


Just to help with readability, when I encode I get :

eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLCAgImNvbmRpdGlvbnMiOiBbICAgIHsiYnVja2V0IjogInNpZ3Y0ZXhhbXBsZWJ1Y2tldCJ9LCAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwgICAgeyJhY2wiOiAicHVibGljLXJlYWQifSwgICAgeyJzdWNjZXNzX2FjdGlvbl9yZWRpcmVjdCI6ICJodHRwOi8vc2lndjRleGFtcGxlYnVja2V0LnMzLmFtYXpvbmF3cy5jb20vc3VjY2Vzc2Z1bF91cGxvYWQuaHRtbCJ9LCAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sICAgIHsieC1hbXotbWV0YS11dWlkIjogIjE0MzY1MTIzNjUxMjc0In0sICAgIHsieC1hbXotc2VydmVyLXNpZGUtZW5jcnlwdGlvbiI6ICJBRVMyNTYifSwgICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sICAgIHsieC1hbXotY3JlZGVudGlhbCI6ICJBS0lBSU9TRk9ETk43RVhBTVBMRS8yMDE1MTIyOS91cy1lYXN0LTEvczMvYXdzNF9yZXF1ZXN0In0sICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwgICAgeyJ4LWFtei1kYXRlIjogIjIwMTUxMjI5VDAwMDAwMFoiIH0gIF19

But I should be getting :

eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLA0KICAiY29uZGl0aW9ucyI6IFsNCiAgICB7ImJ1Y2tldCI6ICJzaWd2NGV4YW1wbGVidWNrZXQifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwNCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LA0KICAgIHsic3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiOiAiaHR0cDovL3NpZ3Y0ZXhhbXBsZWJ1Y2tldC5zMy5hbWF6b25hd3MuY29tL3N1Y2Nlc3NmdWxfdXBsb2FkLmh0bWwifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sDQogICAgeyJ4LWFtei1tZXRhLXV1aWQiOiAiMTQzNjUxMjM2NTEyNzQifSwNCiAgICB7IngtYW16LXNlcnZlci1zaWRlLWVuY3J5cHRpb24iOiAiQUVTMjU2In0sDQogICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sDQoNCiAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlPU0ZPRE5ON0VYQU1QTEUvMjAxNTEyMjkvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LA0KICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwNCiAgICB7IngtYW16LWRhdGUiOiAiMjAxNTEyMjlUMDAwMDAwWiIgfQ0KICBdDQp9

I'm hoping a fresh set of eyes will point me in the direction. Many thanks for any advice that can be offered.

Answer

The following worked for me in Java. Leave the carriage return and line feeds in place.

String policy_document = "{ \"expiration\": \"2015-12-30T12:00:00.000Z\",\n" +
            "  \"conditions\": [\n" +
            "    {\"bucket\": \"sigv4examplebucket\"},\n" +
            "    [\"starts-with\", \"$key\", \"user/user1/\"],\n" +
            "    {\"acl\": \"public-read\"},\n" +
            "    {\"success_action_redirect\": \"http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html\"},\n" +
            "    [\"starts-with\", \"$Content-Type\", \"image/\"],\n" +
            "    {\"x-amz-meta-uuid\": \"14365123651274\"},\n" +
            "    {\"x-amz-server-side-encryption\": \"AES256\"},\n" +
            "    [\"starts-with\", \"$x-amz-meta-tag\", \"\"],\n" +
            "\n" +
            "    {\"x-amz-credential\": \"AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request\"},\n" +
            "    {\"x-amz-algorithm\": \"AWS4-HMAC-SHA256\"},\n" +
            "    {\"x-amz-date\": \"20151229T000000Z\" }\n" +
            "  ]\n" +
            "}";
    String base64Policy = (new BASE64Encoder()).encode(policy_document.getBytes());