Pavel Bernshtam Pavel Bernshtam - 4 months ago 121
JSON Question

Avro: deserialize json - schema with optional fields

There are a lot of questions and answers on stackoverflow on the subject, but no one that helps.

I have a schema with optional value:

{
"type" : "record",
"name" : "UserSessionEvent",
"namespace" : "events",
"fields" : [ {
"name" : "username",
"type" : "string"
}, {
"name" : "errorData",
"type" : [ "null", "string" ],
"default" : null
}]
}


And I'm trying deserialize json w/o this field:

{
"username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
"errorData" : { "string":"070226AC-9B91-47CE-85FE-15AA17972298"}
}


using code:

val reader = new GenericDatumReader[GenericRecord](schema)
val decoder = DecoderFactory.get().jsonDecoder(schema, json)
reader.read(null, decoder)


and I got:
org.apache.avro.AvroTypeException: Expected field name not found: errorData


The only way that works is json

{
"username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
"errorData" : null


}

Is there a way to deserialize json w/o this field?

Another question: when this field is here, I should write

{
"username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
"errorData" : { "string":"070226AC-9B91-47CE-85FE-15AA17972298"}


}

Is there a way to deserialize a "normal" json:

{
"username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
"errorData" : "070226AC-9B91-47CE-85FE-15AA17972298"
}


?

Answer

case 1 is working fine in java .

{
"username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
"errorData" : { "string":"070226AC-9B91-47CE-85FE-15AA17972298"}
}

for case 2 Your schema is defined for union. You can update you schema as below to deserialize json.

 {
   "username" : "2271AE67-34DE-4B43-8839-07216C5D10E1",
   "errorData" : "070226AC-9B91-47CE-85FE-15AA17972298"
 }

{
  "type" : "record",
  "name" : "UserSessionEvent",
  "namespace" : "events",
  "fields" : [ {
                "name" : "username",
                "type" : "string"
             }, {
                "name" : "errorData",
                "type" :  "string" ,
                "default" : null
               }]
}