avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <sc...@richrelevance.com>
Subject Re: java.lang.NoSuchMethodError: org.codehaus.jackson.JsonFactory.enable
Date Tue, 25 Jan 2011 00:25:54 GMT
A jar prior to the jackson jar contains an earlier version of jackson inside of it.

Pig's jar typically contains all its dependencies (there is 'pig-withouthadoop.jar' instead).

So my guess is that one of these (look at a listing of jar contents) has jackson in it:

/usr/lib/pig/bin/../pig-0.7.0+16-core.jar
/usr/lib/pig/bin/../pig-0.7.0+16.jar
/usr/lib/pig/bin/../build/pig-*-core.jar:
/usr/lib/pig/bin/../build/ivy/lib/Pig/*.jar


On 1/24/11 4:06 PM, "felix gao" <gre1600@gmail.com<mailto:gre1600@gmail.com>>
wrote:

here is the actual process that is running the pig script, I hope this helps.

root     20820 19838 66 18:57 pts/0    00:00:00 /usr/java/default/bin/java -Xmx1000m -Djava.library.path=/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl
-Dpig.log.dir=/usr/lib/pig/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/lib/pig/bin/..
-Dpig.root.logger=INFO,console,DRFA -classpath /usr/lib/pig/bin/../conf:/usr/java/default/lib/tools.jar:/usr/lib/pig/bin/../pig-0.7.0+16-core.jar:/usr/lib/pig/bin/../pig-0.7.0+16.jar:/usr/lib/pig/bin/../build/pig-*-core.jar:/usr/lib/pig/bin/../lib/automaton.jar:/usr/lib/pig/bin/../lib/hbase-0.20.0-test.jar:/usr/lib/pig/bin/../lib/hbase-0.20.0.jar:/usr/lib/pig/bin/../lib/zookeeper-hbase-1329.jar:/usr/lib/pig/bin/../build/ivy/lib/Pig/*.jar:/usr/lib/hadoop-0.20/hadoop-core-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-3.1.1.jar:/usr/lib/hadoop-0.20/lib/hadoop-fairscheduler-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.6.jar:/usr/lib/hadoop-0.20/lib/hadoop-thriftfs-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.LICENSE.txt:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jdiff:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-6.1.14.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.14.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.LICENSE.txt:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/mysql-connector-java-5.0.8-bin.jar:/usr/lib/hadoop-0.20/lib/native:/usr/lib/hadoop-0.20/lib/native_libs.tar:/usr/lib/hadoop-0.20/lib/oro-2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo.0.4.4.jar:/usr/lib/hadoop-0.20/conf::/home/felix/hadoop-lzo.jar:/home/felix/elephant-bird.jar:/home/felix/elephant-bird/lib/*
org.apache.pig.Main avro.pig


pig -secretDebugCmd
dry run:
/usr/java/default/bin/java -Xmx1000m -Djava.library.path=/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl
-Dpig.log.dir=/usr/lib/pig/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/lib/pig/bin/..
-Dpig.root.logger=INFO,console,DRFA -classpath /usr/lib/pig/bin/../conf:/usr/java/default/lib/tools.jar:/usr/lib/pig/bin/../pig-0.7.0+16-core.jar:/usr/lib/pig/bin/../pig-0.7.0+16.jar:/usr/lib/pig/bin/../build/pig-*-core.jar:/usr/lib/pig/bin/../lib/automaton.jar:/usr/lib/pig/bin/../lib/hbase-0.20.0-test.jar:/usr/lib/pig/bin/../lib/hbase-0.20.0.jar:/usr/lib/pig/bin/../lib/zookeeper-hbase-1329.jar:/usr/lib/pig/bin/../build/ivy/lib/Pig/*.jar:/usr/lib/hadoop-0.20/hadoop-core-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-3.1.1.jar:/usr/lib/hadoop-0.20/lib/hadoop-fairscheduler-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.6.jar:/usr/lib/hadoop-0.20/lib/hadoop-thriftfs-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.LICENSE.txt:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jdiff:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-6.1.14.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.14.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.LICENSE.txt:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/mysql-connector-java-5.0.8-bin.jar:/usr/lib/hadoop-0.20/lib/native:/usr/lib/hadoop-0.20/lib/native_libs.tar:/usr/lib/hadoop-0.20/lib/oro-2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo.0.4.4.jar:/usr/lib/hadoop-0.20/conf::/home/felix/hadoop-lzo.jar:/home/felix/elephant-bird.jar:/home/felix/elephant-bird/lib/*
org.apache.pig.Main


seems the jackson 1.5.5 is on the classpath of the pig as well as tasktracker and the actual
job.

Felix


On Mon, Jan 24, 2011 at 3:44 PM, Tatu Saloranta <tsaloranta@gmail.com<mailto:tsaloranta@gmail.com>>
wrote:
On Mon, Jan 24, 2011 at 3:13 PM, Scott Carey <scott@richrelevance.com<mailto:scott@richrelevance.com>>
wrote:
> That is confusing.  Can you capture the classpath of an actual task process,
> not just the TT?  They shouldn't differ much, but it is worth checking.
> Jackson 1.3 (or was it 1.2?) and above have all been backwards compatible
> with each other I believe.   And the error you are getting is definitely
> caused by accessing the enable() methods that were added after 1.0.1.
> I can change the Avro dependency on Jackson to 1.5.5, 1.7.1, or 1.3, and
> unit tests pass.  If I change it to 1.2, 1.1, or 1.0.1 they break.

Just in case anyone is interested, this is due to change in 1.3.0
which changed return type of configuration method from 'void' to
ObjectMapper, to allow fluent-style chaining of configuration. This is
source compatible, but unfortunately binary incompatible change. On
plus side, it is the only known such problem, which makes it easier to
recognize.

-+ Tatu +-


Mime
View raw message