avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tazan007 <tazan...@gmail.com>
Subject Re: Unknown type java.util.List<java.lang.String> with ReflectData.getSchema
Date Thu, 06 Aug 2009 23:53:20 GMT
Yea makes sense.  I am trying to go from Java to Avro :D so I ended up
overriding getArraySize, getArrayElements, writeRecord, and writeString from
ReflectDatumWriter so I could have it convert String to Utf8, Date to Long,
and List to GenericArray.

Also had to make some changes to ReflectData's createSchema to support
String and Lists  and changed RecordSchema's fieldsToJson to translate Date
objects to long.  Basically a cluge but works for getting data out of my
Java objects into Avro objects.

If anyone is interested in doing the same, let me know.

-Hiral

On Thu, Aug 6, 2009 at 2:27 PM, Doug Cutting <cutting@apache.org> wrote:

> Sorry, List<String> is not supported, rather one must use
> GenericArray<Utf8>.  The reason is that both the specific and the reflect
> implementations inherit from the generic implementation, primarily just
> overriding the implementation of records.  So the mapping from Avro to Java
> (excepting records) is described at:
>
>
> http://hadoop.apache.org/avro/docs/current/api/java/org/apache/avro/generic/package-summary.html
>
> The GenericArray interface is mostly a subset of the List interface, but
> includes one non-List method, peek(), in order to permit reuse of instances.
>
> Doug
>
> tazan007 wrote:
>
>> Still getting similar error with Utf8:
>>
>> ptype = java.util.List<org.apache.avro.util.Utf8> raw = interface
>> java.util.List
>> param =class org.apache.avro.util.Utf8
>> Exception in thread "main" org.apache.avro.AvroTypeException: Unknown
>> type: java.util.List<org.apache.avro.util.Utf8>
>>    at
>> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:192)
>>    at
>> org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:185)
>>    at org.apache.avro.reflect.ReflectData.getMessage(ReflectData.java:230)
>>    at
>> org.apache.avro.reflect.ReflectData.getProtocol(ReflectData.java:206)
>>
>> Are lists not support?
>>
>> -Hiral
>>
>> On Fri, Jul 31, 2009 at 4:56 PM, Doug Cutting <cutting@apache.org<mailto:
>> cutting@apache.org>> wrote:
>>
>>    Avro uses org.apache.avro.util.Utf8 to represent strings, not
>>    java.lang.String.
>>
>>    Doug
>>
>>
>>    tazan007 wrote:
>>
>>        Hi guys, I am getting this error when trying to build a schema
>>        using an existing class with ReflectData.getSchema method with a
>>        List<String> field.  Is this expected behaviour?  Are List's not
>>        supported?
>>
>>        ptype = java.util.List<java.lang.String> raw = interface
>>        java.util.List
>>        param =class java.lang.String
>>        Exception in thread "main" org.apache.avro.AvroTypeException:
>>        Unknown type: java.util.List<java.lang.String>
>>           at
>>
>>  org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:192)
>>           at
>>
>>  org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:185)
>>           at
>>        org.apache.avro.reflect.ReflectData.getSchema(ReflectData.java:107)
>>
>>        Thanks,
>>        Hiral
>>
>>
>>

Mime
View raw message