crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samik Raychaudhuri <sam...@gmail.com>
Subject Re: Getting to PCollection of specific class
Date Thu, 11 Sep 2014 12:54:24 GMT
Ah .. turned out to be a rather easy fix - I had to upgrade avro library 
version from 1.7.4 to the most recent one (1.7.7).
Thanks.

On 11/09/2014 6:11 PM, Samik Raychaudhuri wrote:
> Hello,
> Newbie alert :-)
> I am trying to get a PCollection of the class which is a POJO 
> generated from my avro schema. Here is what I have:
>
> AvroMode.setSpecificClassLoader(Event.class.getClassLoader());
>         PCollection<Event> records = 
> pipeline.read(From.avroFile("/raw/*.avro", Avros.specifics(Event.class)));
>         pipeline.materialize(records);
>
> The Event class above is generated from the avro schema. I am getting 
> the following exception:
>
> Exception in thread "main" java.lang.NoSuchMethodError: 
> org.apache.avro.Schema$Parser.parse(Ljava/lang/String;[Ljava/lang/String;)Lorg/apache/avro/Schema;
>     at com.tfs.interaction.event.avro.Event.<clinit>(Event.java:10)
>     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
>     at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>     at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>     at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
>     at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
>     at org.apache.crunch.types.avro.Avros.specifics(Avros.java:267)
> *at 
> com.ilabs.dsi.dataQuality.DataQualityChecker.run(DataQualityChecker.java:50)*
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>     at 
> com.ilabs.dsi.dataQuality.DataQualityChecker.main(DataQualityChecker.java:81)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:483)
>     at 
> com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
>
> The line from where the Exception starts is the second line in the 
> snippet above. I essentially tried reusing the snippet that is 
> mentioned here: https://issues.apache.org/jira/browse/CRUNCH-442
>
> When I load the data using the following snippet to Record objects, 
> things work fine:
>
>         Schema eventSchema = new Schema.Parser().parse(new 
> File(schemaPath));
>         System.out.println("Loaded IDM schema.");
>         PCollection<Record> events = 
> pipeline.read(From.avroFile("/raw/*.avro", Avros.generics(eventSchema)));
>
>
> What am I doing wrong?
> Regards.


Mime
View raw message