avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <sc...@richrelevance.com>
Subject Re: Can I nest schema?
Date Mon, 16 May 2011 22:22:50 GMT
You can dynamically create a record for this job:

Schema.createRecord( … )
create a field with the int,
create a field with the Record,
put these in a List,
call setFields() on the record.

Use that record for the job.

The result is a record with two fields, the int and the nested Record.

On 5/16/11 3:10 PM, "W.P. McNeill" <billmcn@gmail.com<mailto:billmcn@gmail.com>>

I am writing a Hadoop application whose values are objects called Records which are serialized
using Avro.  (I specify a Serialization class for the Records via the io.serializations property.)

I now need to expand my application so that instead of just a Record I need to have a more
complicated data structure, call it an Augmented Record.  Say that an Augmented Record contains
integer N in addition to the record, so now the value looks like (N, Record).  Adding an integer
field to the Record schema just to support this one Hadoop process would be a hack, but I
also can't create a Writable (WritableInt, Record) object because Record uses its own Avro
serialization scheme and so is not Writable.  What I want to do is basically create a new
schema of the form [Integer: N, Record: R], where the Record schema is read in dynamically.
 Can I dynamically nest schema in this manner?  If not, what is the best approach to serializing
an Augmented Record?


View raw message