avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Robertson <timrobertson...@gmail.com>
Subject Re: Default values not supported in the client write?
Date Mon, 02 Aug 2010 18:58:29 GMT
Thanks for the explanations,  I'll go for wrappers but wanted to get
some advice first.

So am I to understand that there is no way of having optional values
as you'd get the following?

Exception in thread "main" java.lang.NullPointerException
       at org.apache.avro.io.BinaryEncoder.writeString(BinaryEncoder.java:133)
       at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:176)

So for an example I would need to put a NULL in an Enum to allow for
explicit indication of the Enum not being set?
I am only using the example code so maybe I am using the wrong writers
and should be doing something more specific...

Thanks,
Tim



On Mon, Aug 2, 2010 at 8:06 PM, Scott Carey <scott@richrelevance.com> wrote:
> Right now, defaults are for readers to read data written with a differing schema.  This
is for schema evolution, not for making writing easier.
>
> We could add a feature to a DatumWriter in the Java API to set defaults during writing
if the field is null, or a method on a SpcificRecord that will set defaults.
> However, there is some safety provided by this -- it catches users who forget to set
a field before it is written incorrectly.
>
> I recommend creating a wrapper class or static helpers that handle your type construction
and/or field manipulation.  This does two things:  It abstracts your code from the exact
SpecificRecord implementation details, and provides you with the ability to control specific
requirements for construction and use of your objects.  In this way you can control how truly
required fields are differentiated from truly optional fields.
>
> "default" in avro is not there to distinguish 'optional' from 'required' fields -- it
is there to manage schema evolution.
>
> On Aug 2, 2010, at 9:51 AM, Tim Robertson wrote:
>
>> Hi all,
>>
>> I am doing the protocol compilation to Java using the maven plugin and
>> looking for an explanation on the default values please.
>>
>> http://www.mail-archive.com/avro-dev@hadoop.apache.org/msg04063.html
>> tells me that defaults aren't used at write time, but it means that I
>> have to initiate all the Strings with "=new Utf8();" each time I
>> create a message payload - is this correct?
>>
>> If I set them to null, then I always see:
>> Exception in thread "main" java.lang.NullPointerException
>>       at org.apache.avro.io.BinaryEncoder.writeString(BinaryEncoder.java:133)
>>       at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:176)
>>
>> I wonder the reasoning behind not initializing with null types (e.g.
>> for String using new Utf8()) in the client generation.  It means I
>> either need to modify the generated sources, or wrap them in some kind
>> of factory/builder right?  Is that a common practice for avro or are
>> others doing something different?
>>
>> Any recommendations for best use greatly appreciated.  Currently in my
>> working test I have to:
>>
>>                       Event e = new Event();
>>                       e.level=LEVEL_TYPES.DEBUG;  // no auto ENUM default?
>>                       e.objectType = OBJECT_TYPES.NAME_USAGE;
>>                       e.sourceComponentType=SOURCE_COMPONENT_TYPES.CHECKLIST_HARVESTER;
>>                       e.agentId=new Utf8("Hi list");
>>                       e.count=0;
>>                       e.instanceId=new Utf8("127.0.0.1:7004");
>>                       e.message=new Utf8();
>>                       e.objectId=new Utf8();
>>                       //e.sensitive=false;  // works because is a primitive
>>                       e.sessionId=new Utf8("dysAw231");
>>                       e.sourceId=new Utf8();
>>                       e.userId=new Utf8();
>>                       logger.info(e, false);
>>
>>
>> Thanks,
>> Tim
>
>

Mime
View raw message