crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samik Raychaudhuri <sam...@gmail.com>
Subject Getting to PCollection of specific class
Date Thu, 11 Sep 2014 12:41:45 GMT
Hello,
Newbie alert :-)
I am trying to get a PCollection of the class which is a POJO generated 
from my avro schema. Here is what I have:

AvroMode.setSpecificClassLoader(Event.class.getClassLoader());
         PCollection<Event> records = 
pipeline.read(From.avroFile("/raw/*.avro", Avros.specifics(Event.class)));
         pipeline.materialize(records);

The Event class above is generated from the avro schema. I am getting 
the following exception:

Exception in thread "main" java.lang.NoSuchMethodError: 
org.apache.avro.Schema$Parser.parse(Ljava/lang/String;[Ljava/lang/String;)Lorg/apache/avro/Schema;
     at com.tfs.interaction.event.avro.Event.<clinit>(Event.java:10)
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
     at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
     at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
     at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
     at org.apache.crunch.types.avro.Avros.specifics(Avros.java:267)
*at 
com.ilabs.dsi.dataQuality.DataQualityChecker.run(DataQualityChecker.java:50)*
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at 
com.ilabs.dsi.dataQuality.DataQualityChecker.main(DataQualityChecker.java:81)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:483)
     at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

The line from where the Exception starts is the second line in the 
snippet above. I essentially tried reusing the snippet that is mentioned 
here: https://issues.apache.org/jira/browse/CRUNCH-442

When I load the data using the following snippet to Record objects, 
things work fine:

         Schema eventSchema = new Schema.Parser().parse(new 
File(schemaPath));
         System.out.println("Loaded IDM schema.");
         PCollection<Record> events = 
pipeline.read(From.avroFile("/raw/*.avro", Avros.generics(eventSchema)));


What am I doing wrong?
Regards.

Mime
View raw message