avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ayyappan Arasu <ayyap...@yahoo-inc.com>
Subject Support for null in String primitive types
Date Tue, 27 Nov 2012 18:11:09 GMT
Hi,

I have an avro schema that I would like to use the default values as null for string types(primitives).
Is there a way I can enable it or is this intentional? I am getting following error -

Error>

FAILED: testAvroToJson
org.apache.avro.AvroTypeException: Non-string default value for string: null
       at org.apache.avro.io.parsing.ResolvingGrammarGenerator.encode(ResolvingGrammarGenerator.java:363)
       at org.apache.avro.data.RecordBuilderBase.defaultValue(RecordBuilderBase.java:178)

Avro Schema>

{
       "schemas" : [
              {
                     "namespace" : "com.mycompany",
                     "version" : "1.0",
                     "name" : "employee",
                     "type" : "record",
                     "fields" : [
                     {"name" : " _id", "type": "string"},
                     {"name" : "_name", "type" : "string" },
                      {"name" : "_email" , "type": "string", "default" : null }
                     ]
              }
       ]
}

Test Code>

    private String avro2Json(Object avroData){
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        GenericDatumWriter<Object> writer = new GenericDatumWriter<Object>(avroSchema);
        String json = null;
        try {
            JsonEncoder jsonEncoder = EncoderFactory.get().jsonEncoder(avroSchema, output);
            writer.write(avroData, jsonEncoder);
            jsonEncoder.flush();
            json = output.toString();
        } catch (IOException ioe) {
            logger.error("Error occured:" + ioe.getMessage());
        } finally {
            try {
                output.close();
            } catch (IOException e) {
                // ignore
                logger.error("Error occured:" + e.getMessage());
            }
        }
        return json;
    }

    @Test
    public void testAvroToJson(){
        GenericRecordBuilder builder = new GenericRecordBuilder(avroSchema);
        builder.set("_id", "1000")
               .set("_name", "Dustin Hoffman")
        String jsonStr = avro2Json(builder.build());
        JsonNode node = null;
        try{
            node = mapper.readValue(jsonStr, JsonNode.class);
        }catch(JsonParseException jpe){
            Assert.fail("Exception occured when it should not have:" + jpe.getMessage());
        }catch(JsonMappingException jme){
            Assert.fail("Exception occured when it should not have:" + jme.getMessage());
        }catch(IOException ioe){
           Assert.fail("Exception occured when it should not have:" + ioe.getMessage());
        }
        Assert.assertNotNull(node);
        Assert.assertEquals(node.get("_id").getValueAsText(), "1000");
    }

It seems the failure is in   public static void encode(Encoder e, Schema s, JsonNode n) in
org.apache.avro.io.parsing.ResolvingGrammerGenerator

In the check below, the value of n is NullNode. Shouldn't String be nullable by default?

    case STRING:
      if (!n.isTextual())
        throw new AvroTypeException("Non-string default value for string: "+n);
      e.writeString(n.getTextValue());
      break;

Please let me know if you have any pointers around this?

Thanks,
Ayyappan


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message