avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: expect specific record but get generic
Date Mon, 21 Oct 2013 20:49:29 GMT
On Mon, Oct 21, 2013 at 1:19 PM, Koert Kuipers <koert@tresata.com> wrote:
> doug, could it be a classloader (instead of classpath) issue? looking at
> spark it seems to run the tasks inside the slaves/workers with a custom
> classloader.

Yes, it could be a classloader issue.  Perhaps you need to pass the
Spark task's ClassLoader to SpecificData?  Find where your
SpecificDatumReader is constructed, and pass it a SpecificData that
was constructed with that ClassLoader.

http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificData.html#SpecificData(java.lang.ClassLoader)
http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificDatumReader.html#SpecificDatumReader(org.apache.avro.specific.SpecificData)

Doug

Mime
View raw message