avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From felix gao <gre1...@gmail.com>
Subject Re: java.lang.NoSuchMethodError: org.codehaus.jackson.JsonFactory.enable
Date Mon, 24 Jan 2011 22:40:34 GMT
sorry for the spamming, but I forgot to include the TaskTracker's process
information.

root     20368     1  0 17:31 pts/0    00:00:00 su mapred -s
/usr/java/default/bin/java -- -Dproc_tasktracker -Xmx1000m
-Dhadoop.log.dir=/usr/lib/hadoop-0.20/logs
-Dhadoop.log.file=hadoop-hadoop-tasktracker-ip-10-212-86-214.log
-Dhadoop.home.dir=/usr/lib/hadoop-0.20 -Dhadoop.id.str=hadoop
-Dhadoop.root.logger=INFO,DRFA
-Djava.library.path=/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64:/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64
-Dhadoop.policy.file=hadoop-policy.xml -classpath
/etc/hadoop-0.20/conf:/usr/java/default/lib/tools.jar:/usr/lib/hadoop-0.20:/usr/lib/hadoop-0.20/hadoop-core-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-3.1.1.jar:/usr/lib/hadoop-0.20/lib/hadoop-fairscheduler-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.6.jar:/usr/lib/hadoop-0.20/lib/hadoop-thriftfs-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.jar:
/usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-6.1.14.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.14.jar:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/mysql-connector-java-5.0.8-bin.jar:/usr/lib/hadoop-0.20/lib/oro-2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-api-2.1.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo.0.4.4.jar
org.apache.hadoop.mapred.TaskTracker

as you can see the correct version of jackson library is in the classpath.


On Mon, Jan 24, 2011 at 2:09 PM, felix gao <gre1600@gmail.com> wrote:

> Chase,
>
> I tried to ran it on my local box with a standalone version of hadoop
> installed and I still got the same error.
> /usr/local/pig-0.7.0/bin/pig avro.pig
> 11/01/24 14:01:29 INFO pig.Main: Logging error messages to:
> /Users/felix/Documents/pig/pig_1295906489023.log
> 2011-01-24 14:01:29,328 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at: file:///
> 2011-01-24 14:01:30,277 [main] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 2011-01-24 14:01:30,537 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2998: Unhandled internal error.
> org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
>
> Any ideas why?
>
> Thanks,
>
> Felix
>
>
>
>
> On Mon, Jan 24, 2011 at 1:18 PM, Chase Bradford <chase.bradford@gmail.com>wrote:
>
>> Yes, you will need to restart them.  The child tasks inherit the TT's
>> classpath, which will list only the 1.0.1 jars until you restart the
>> daemon.
>>
>> On Mon, Jan 24, 2011 at 1:11 PM, felix gao <gre1600@gmail.com> wrote:
>> > Hi Guys,
>> >
>> > I am testing out AVRO in our cluster and having
>> java.lang.NoSuchMethodError:
>> >
>> org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
>> > when running a simple pig script.
>> > after took a look at AVRO-493 I  uploaded the new jackson jar to replace
>> the
>> > CDH2's jackson 1.0.1 jars.
>> >
>> >
>> > The pigscipt looks like below
>> > REGISTER /home/pig/jars/avro-1.4.1.jar
>> > REGISTER /home/pig/jars/json_simple-1.1.jar
>> > REGISTER /home/pig/jars/piggybank.jar
>> > REGISTER /usr/lib/hadoop/lib/jackson-core-asl-1.5.5.jar
>> > REGISTER /usr/lib/hadoop/lib/jackson-mapper-asl-1.5.5.jar
>> >
>> > log_load = LOAD '/user/felix/avro_input/*.avro' USING
>> > org.apache.pig.piggybank.storage.avro.AvroStorage() ;
>> >
>> > dump log_load
>> >
>> >
>> > jackson jars are copied to each slaves and master
>> >
>> > The full stacktrace:
>> > ERROR 2998: Unhandled internal error.
>> >
>> org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
>> >
>> > java.lang.NoSuchMethodError:
>> >
>> org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
>> >         at org.apache.avro.Schema.<clinit>(Schema.java:82)
>> >         at
>> >
>> org.apache.pig.piggybank.storage.avro.ASCommons.<clinit>(ASCommons.java:44)
>> >         at
>> >
>> org.apache.pig.piggybank.storage.avro.AvroStorage.getSchema(AvroStorage.java:177)
>> >         at
>> >
>> org.apache.pig.piggybank.storage.avro.AvroStorage.getAvroSchema(AvroStorage.java:133)
>> >         at
>> >
>> org.apache.pig.piggybank.storage.avro.AvroStorage.getAvroSchema(AvroStorage.java:108)
>> >         at
>> >
>> org.apache.pig.piggybank.storage.avro.AvroStorage.getSchema(AvroStorage.java:233)
>> >         at
>> > org.apache.pig.impl.logicalLayer.LOLoad.determineSchema(LOLoad.java:169)
>> >         at
>> > org.apache.pig.impl.logicalLayer.LOLoad.getSchema(LOLoad.java:150)
>> >         at
>> >
>> org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:843)
>> >         at
>> >
>> org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>> >         at
>> org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1164)
>> >         at
>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1114)
>> >         at org.apache.pig.PigServer.registerQuery(PigServer.java:425)
>> >         at
>> > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:737)
>> >         at
>> >
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324)
>> >         at
>> >
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162)
>> >         at
>> >
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
>> >         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
>> >         at org.apache.pig.Main.main(Main.java:357)
>> >
>> > I am wondering if I need to restart tasktracker and jobtracker in order
>> for
>> > the jackson jars to be picked up?
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Chase Bradford
>>
>>
>> “If in physics there's something you don't understand, you can always
>> hide behind the uncharted depths of nature. But if your program
>> doesn't work, there is no obstinate nature. If it doesn't work, you've
>> messed up.”
>>
>> - Edsger Dijkstra
>>
>
>

Mime
View raw message