avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <sc...@richrelevance.com>
Subject Re: Need help with default in avro
Date Thu, 04 Nov 2010 21:12:58 GMT
I suspect that you need to inform the GenericDatumReader what the "actual" versus "expected"
schemas are.

The stack trace below has ValidatingDecoder in it, but not ResolvingDecoder.  ResolvingDecoder
is what populates defaults.

GenericDatumReader has two methods for the schema:

setSchema() and setExpected()

expected is the "reader's" schema.  The actual schema of the data is set via setSchema() and
is sometimes referred to as the "writer's" schema.

When GenericDatumReader has its actual and expected schemas set differently, it activates
a ResolvingDecoder to handle making the written schema data appear like the reader's expected
schema.

-Scott



On Nov 4, 2010, at 10:23 AM, Tejal Khot wrote:



Hi,

I am new to avro and I have been trying to resolve the AvroTypeException when using “default”
with the avro schema:
In the schema below, the service_wait field is a Boolean and I want to specify a default value.
However, I keep getting the exception below. What is the correct way to specify defaults?

org.apache.avro.AvroTypeException: Expected field name service_wait got select_branch
            at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:396)
            at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)

Sample Schema:

{
  "type": "record",
  "name": "TestRequest",
  "fields" : [
    {"name": "ver", "type": "string"},
    {"name": "trace_id", "type": "int"},
    {"name": "error_at", "type": "string"},
    {"name": "exception_at", "type": "string"},
    {"name": "bytes", "type": "bytes"},
    {"name": "need_trace_buffer", "type": "boolean"},
    {"name": "service_wait", "type": "boolean", "default" :"true"},
    {"name": "select_branch", "type": "string"},
    {
      "name": "serviceData",
      "type": {
        "type": "array",
        "items":  {
          "type": "record",
          "name": "ServiceData",
          "fields" : [
            {"name": "nm", "type": "string"},
            {"name": "latency", "type": "int"},
            {"name": "req", "type": "bytes"}
          ]
        }
      }
    }


  ]
}

Sample data file is as below:
{
    "ver" : "1.0",
    "trace_id" : 3000301,
    "error_at" : "",
    "exception_at" : "",
    "bytes" : "1234567" ,
    "need_trace_buffer" : false,
    "service_wait" : true,
    "select_branch" : "",
    "serviceData" : [
        {
            "nm" : "HttpService3",
            "latency" : 5,
            "req" : "/"
        }
    ]
}



The exception:

org.apache.avro.AvroTypeException: Expected field name service_wait got select_branch
            at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:396)
            at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
            at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:67)
            at org.apache.avro.io.JsonDecoder.readBoolean(JsonDecoder.java:96)
            at org.apache.avro.io.ValidatingDecoder.readBoolean(ValidatingDecoder.java:63)
            at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:92)
            at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:108)
            at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:80)
            at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:71)
            at com.yahoo.dataintensive.streaming.streaming2transporter.AvroSchemaTest1.main(AvroSchemaTest1.java:32)


Thanks,
Tejal


Mime
View raw message