hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <sc...@richrelevance.com>
Subject Re: Avro, Hadoop0.20.2, Jackson Error
Date Tue, 27 Mar 2012 03:19:15 GMT
Does it still happen if you configure

avro-tools to use 

<dependency>
  <groupId>org.apache.avro</groupId>
  <artifactId>avro-tools</artifactId>
  <version>1.6.3</version>
  <classifier>nodeps</classifier>
    </dependency>


?

You have two hadoop's, two jacksons, and even two avro:avro artifacts in
your classpath if you use the avro bundle jar with a default classifier.

avro-tools jar is not intended for inclusion in a project, as it is a jar
with dependencies inside.
https://cwiki.apache.org/confluence/display/AVRO/Build+Documentation#BuildD
ocumentation-ProjectStructure

On 3/26/12 7:52 PM, "Deepak Nettem" <deepaknettem@gmail.com> wrote:

>When I include some Avro code in my Mapper, I get this error:
>
>Error:
>org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$F
>eature;)Lorg/codehaus/jackson/JsonFactory;
>
>Particularly, just these two lines of code:
>
>            InputStream in =
>getClass().getResourceAsStream("schema.avsc");
>            Schema schema = Schema.parse(in);
>
>This code works perfectly when run as a stand alone application outside of
>Hadoop. Why do I get this error? and what's the best way to get rid of it?
>
>I am using Hadoop 0.20.2, and writing code in the new API.
>
>I found that the Hadoop lib directory contains jackson-core-asl-1.0.1.jar
>and jackson-mapper-asl-1.0.1.jar.
>
>I removed these, but got this error:
>hadoop Exception in thread "main" java.lang.
>NoClassDefFoundError: org/codehaus/jackson/map/JsonMappingException
>
>I am using Maven as a build tool, and my pom.xml has this dependency:
>
>    <dependency>
>    <groupId>org.codehaus.jackson</groupId>
>      <artifactId>jackson-mapper-asl</artifactId>
>      <version>1.5.2</version>
>      <scope>compile</scope>
>    </dependency>
>
>
>------------
>
>I added the dependency:
>
>
><dependency>
>    <groupId>org.codehaus.jackson</groupId>
>      <artifactId>jackson-core-asl</artifactId>
>      <version>1.5.2</version>
>      <scope>compile</scope>
>    </dependency>
>
>But that still gives me this error:
>
>Error: org.codehaus.jackson.
>JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus
>/jackson/JsonFactory;
>
>-------------
>
>I also tried replacing the earlier dependencies with these:
>
>   <dependency>
>    <groupId>org.apache.avro</
>groupId>
>    <artifactId>avro-tools</artifactId>
>    <version>1.6.3</version>
>    </dependency>
>
>    <dependency>
>    <groupId>org.apache.avro</groupId>
>    <artifactId>avro</artifactId>
>    <version>1.6.3</version>
>    </dependency>
>
>
>    <dependency>
>    <groupId>org.codehaus.jackson</groupId>
>      <artifactId>jackson-mapper-asl</artifactId>
>      <version>1.8.8</version>
>      <scope>compile</scope>
>    </dependency>
>
>    <dependency>
>    <groupId>org.codehaus.jackson</groupId>
>      <artifactId>jackson-core-asl</artifactId>
>      <version>1.8.8</version>
>      <scope>compile</scope>
>    </dependency>
>
>And this is my app dependency tree:
>
>[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ AvroTest ---
>[INFO] org.avrotest:AvroTest:jar:1.0-SNAPSHOT
>[INFO] +- junit:junit:jar:3.8.1:test (scope not updated to compile)
>[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
>[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
>[INFO] +- net.sf.json-lib:json-lib:jar:jdk15:2.3:compile
>[INFO] |  +- commons-beanutils:commons-beanutils:jar:1.8.0:compile
>[INFO] |  +- commons-collections:commons-collections:jar:3.2.1:compile
>[INFO] |  +- commons-lang:commons-lang:jar:2.4:compile
>[INFO] |  +- commons-logging:commons-logging:jar:1.1.1:compile
>[INFO] |  \- net.sf.ezmorph:ezmorph:jar:1.0.6:compile
>[INFO] +- org.apache.avro:avro-tools:jar:1.6.3:compile
>[INFO] |  \- org.slf4j:slf4j-api:jar:1.6.4:compile
>[INFO] +- org.apache.avro:avro:jar:1.6.3:compile
>[INFO] |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
>[INFO] |  \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
>[INFO] \- org.apache.hadoop:hadoop-core:jar:0.20.2:compile
>[INFO]    +- commons-cli:commons-cli:jar:1.2:compile
>[INFO]    +- xmlenc:xmlenc:jar:0.52:compile
>[INFO]    +- commons-httpclient:commons-httpclient:jar:3.0.1:compile
>[INFO]    +- commons-codec:commons-codec:jar:1.3:compile
>[INFO]    +- commons-net:commons-net:jar:1.4.1:compile
>[INFO]    +- org.mortbay.jetty:jetty:jar:6.1.14:compile
>[INFO]    +- org.mortbay.jetty:jetty-util:jar:6.1.14:compile
>[INFO]    +- tomcat:jasper-runtime:jar:5.5.12:compile
>[INFO]    +- tomcat:jasper-compiler:jar:5.5.12:compile
>[INFO]    +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile
>[INFO]    +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile
>[INFO]    |  \- ant:ant:jar:1.6.5:compile
>[INFO]    +- commons-el:commons-el:jar:1.0:compile
>[INFO]    +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
>[INFO]    +- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile
>[INFO]    +- net.sf.kosmosfs:kfs:jar:0.3:compile
>[INFO]    +- hsqldb:hsqldb:jar:1.8.0.10:compile
>[INFO]    +- oro:oro:jar:2.0.8:compile
>[INFO]    \- org.eclipse.jdt:core:jar:3.1.1:compile
>
>I still get the same error.
>
>Somebody please please help me with this. I need to resolve this asap!!
>
>Best,
>Deepak


Mime
View raw message