avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Wolf <opus...@gmail.com>
Subject Re: Schema with multiple Record types Java API
Date Fri, 15 Jul 2011 11:54:17 GMT
Thanks again Scott,

Yes, I am using AVRO to serialize existing Java classes, so tools to 
generate code will not help me.

Are there tools that go the other way, such as JAXB for XML?  I really 
want to point to a root Java object, and say "serialize this, and 
everything it points to, as AVRO".

BTW AVRO Rocks!  My objects contain are amounts of data, and I am *very* 
impressed with the speed of serialization/deserialization.

Cheers
P





On 7/14/11 10:10 PM, Scott Carey wrote:
> AvroIDL can handle imports, but it generates classes.  The Avro API's 
> for this can be used to generate Schemas without making objects if you 
> wish.
>
> The Avro schema compiler (*.avsc, *.avpr) does not support imports, it 
> is a feature requested by many but not contributed by anyone.
>
> You may be interested in the code-gen capabilities of Avro, which has 
> a Velocity templating engine to create Java classes based on schemas. 
>  This can be customized to generate classes in custom ways.
>
> However, if you are using Avro to serialize objects that have 
> pre-existing classes, the Reflect API or an enhancement of it may be 
> more suitable.
>
> More information on your use case may help to point you in the right 
> direction.
>
> -Scott
>
>
> On 7/14/11 6:43 PM, "Peter Wolf" <opus111@gmail.com 
> <mailto:opus111@gmail.com>> wrote:
>
>     Many thanks Scott,
>
>     I am looking for the equivalent of #include or import.  I want to
>     make a complicated schema with many record types, but manage it in
>     separate strings.
>
>     In my application, I am using AVRO to serialize a tree of
>     connected Java objects.  The record types mirror Java classes. 
>     The schema descriptions live in the different Java classes, and
>     reference each other.
>
>     My current code looks like this...
>
>         public class Foo {
>
>             static String schemaDescription =
>                 "{" +
>                         "  \"namespace\": \"foo\", " +
>                         "  \"name\": \"Foo\", " +
>                         "  \"type\": \"record\", " +
>                         "  \"fields\": [ " +
>                         "      {\"name\": \"notes\", \"type\":
>     \"string\" }, " +
>                         "      {\"name\": \"timestamp\", \"type\":
>     \"string\" }, " +
>                         "      {\"name\": \"bah\", \"type\": " +
>     Bah.schemaDescription + " }," +
>                         "      {\"name\": \"zot\", \"type\": " +
>     Zot.schemaDescription + " }" +
>                         "    ]" +
>                         "}";
>
>             static Schema schema = Schema.parse(schemaDescription);
>
>
>     So, I am referencing by copying the schemaDescriptions.  The top
>     level schemaDescription strings therefore get really big.
>
>     Is there already a clean coding Pattern for doing this-- I can't
>     be the first.  Is there a document describing best practices?
>
>     Thanks
>     P
>
>
>
>
>
>     On 7/14/11 7:02 PM, Scott Carey wrote:
>>     The name and namespace is part of any named schema (Type.RECORD,
>>     Type.FIXED, Type.ENUM).
>>
>>     We don't currently have an API to search a schema for subschemas
>>     that match names.  It would be useful, you might want to create a
>>     JIRA ticket explaining your use case.
>>
>>     So it would be a little more complex.
>>
>>             Schema schema = Schema.parse(schemaDescription);
>>             Schema.Type type = schema.getType();
>>             switch (type) {
>>             case RECORD:
>>               String name = schema.getName();
>>               String namespace = schema.getNamespace();
>>               List<Field> fields = schema.getFields();
>>             }
>>             etc.
>>
>>     In general, I have created SpecificRecord objects from schemas
>>     using the specific compiler (and the ant task or maven plugin)
>>     and then within those generated classes there is a static SCHEMA
>>     variable to reference.
>>
>>     Avro IDL is alo an easier way to define related schemas.
>>      Currently there are only build tools that generate code from
>>     these, though there are APIs to extract schemas.
>>
>>     -Scott
>>
>>     On 7/13/11 10:43 AM, "Peter Wolf" <opus111@gmail.com
>>     <mailto:opus111@gmail.com>> wrote:
>>
>>         Hello, this a dumb question, but I can not find the answer in
>>         the docs
>>
>>         I want to have a complicated schema with lots of Records
>>         referencing other Records.
>>
>>         Like this...
>>
>>             {
>>               "namespace": "com.foobah",
>>               "name": "Bah",
>>               "type": "record",
>>               "fields": [
>>               {"name": "value", "type": "int"}
>>               ]
>>             }
>>
>>             {
>>               "namespace": "com.foobah",
>>               "name": "Foo",
>>               "type": "record",
>>               "fields": [
>>               {"name": "bah", "type": "Bah"}
>>               ]
>>             }
>>
>>         Using the Java API, how do I reference types within a
>>         schema?  Let's say I want to make a Foo object, I want to do
>>         something like this...
>>
>>                 Schema schema = Schema.parse(schemaDescription);
>>         >>> Schema foo = schema.getSchema("com.foobah.Foo"); <<<
>>                 GenericData o = new GenericData( foo );
>>
>>         Many thanks in advance
>>         Peter
>>
>>
>>
>


Mime
View raw message