avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Nettem <deepaknet...@gmail.com>
Subject Re: Java MapReduce Avro Jackson Error
Date Tue, 27 Mar 2012 02:58:14 GMT
Hi Folks,

This issue is still not resolved :( Any other ideas?

Best,
Deepak

On Mon, Mar 19, 2012 at 11:25 PM, Scott Carey <scottcarey@apache.org> wrote:

> If you are using avro-tools, beware it is a shaded jar with all
> dependencies inside of it for use as a command line tool (java –jar
> avro-tools-VERSION.jar).
>
> If you are using avro-tools in your project for some reason (there is
> really only command line utilities in it) use the nodeps classifier:
>
> <classifier>nodeps</classifier>
>
> http://repo1.maven.org/maven2/org/apache/avro/avro-tools/1.6.3/
>
> Note the nodeps jar is 47K, while the default jar is 10MB.
>
>
> For what it is worth, I removed the Jackson jar from our hadoop install
> long ago.  It is used to dump configuration files to JSON there, a
> peripheral feature we don't use.
>
> Another thing that you may want to do is change your Hadoop dependency
> scope to
> <scope>provided</scope> since hadoop will be put on your classpath by the
> hadoop environment.   Short of this, excluding the chained Hadoop
> dependencies you aren't using (most likely: jetty,  kfs, and the
> tomcat:jasper and eclipse:jdt stuff) may help.
>
> On 3/19/12 6:23 PM, "Deepak Nettem" <deepaknettem@gmail.com> wrote:
>
> Hi Tatu / Scott,
>
> Thanks for your replies. I replaced the earlier dependencies with these:
>
>    <dependency>
>     <groupId>org.apache.avro</groupId>
>     <artifactId>avro-tools</artifactId>
>     <version>1.6.3</version>
>     </dependency>
>
>     <dependency>
>     <groupId>org.apache.avro</groupId>
>     <artifactId>avro</artifactId>
>     <version>1.6.3</version>
>     </dependency>
>
>     <dependency>
>     <groupId>org.codehaus.jackson</groupId>
>       <artifactId>jackson-mapper-asl</artifactId>
>       <version>1.8.8</version>
>       <scope>compile</scope>
>     </dependency>
>
>     <dependency>
>     <groupId>org.codehaus.jackson</groupId>
>       <artifactId>jackson-core-asl</artifactId>
>       <version>1.8.8</version>
>       <scope>compile</scope>
>     </dependency>
>
> And this is my app dependency tree:
>
> [INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ AvroTest ---
> [INFO] org.avrotest:AvroTest:jar:1.0-SNAPSHOT
> [INFO] +- junit:junit:jar:3.8.1:test (scope not updated to compile)
> [INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
> [INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
> [INFO] +- net.sf.json-lib:json-lib:jar:jdk15:2.3:compile
> [INFO] |  +- commons-beanutils:commons-beanutils:jar:1.8.0:compile
> [INFO] |  +- commons-collections:commons-collections:jar:3.2.1:compile
> [INFO] |  +- commons-lang:commons-lang:jar:2.4:compile
> [INFO] |  +- commons-logging:commons-logging:jar:1.1.1:compile
> [INFO] |  \- net.sf.ezmorph:ezmorph:jar:1.0.6:compile
> [INFO] +- org.apache.avro:avro-tools:jar:1.6.3:compile
> [INFO] |  \- org.slf4j:slf4j-api:jar:1.6.4:compile
> [INFO] +- org.apache.avro:avro:jar:1.6.3:compile
> [INFO] |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
> [INFO] |  \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
> [INFO] \- org.apache.hadoop:hadoop-core:jar:0.20.2:compile
> [INFO]    +- commons-cli:commons-cli:jar:1.2:compile
> [INFO]    +- xmlenc:xmlenc:jar:0.52:compile
> [INFO]    +- commons-httpclient:commons-httpclient:jar:3.0.1:compile
> [INFO]    +- commons-codec:commons-codec:jar:1.3:compile
> [INFO]    +- commons-net:commons-net:jar:1.4.1:compile
> [INFO]    +- org.mortbay.jetty:jetty:jar:6.1.14:compile
> [INFO]    +- org.mortbay.jetty:jetty-util:jar:6.1.14:compile
> [INFO]    +- tomcat:jasper-runtime:jar:5.5.12:compile
> [INFO]    +- tomcat:jasper-compiler:jar:5.5.12:compile
> [INFO]    +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile
> [INFO]    +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile
> [INFO]    |  \- ant:ant:jar:1.6.5:compile
> [INFO]    +- commons-el:commons-el:jar:1.0:compile
> [INFO]    +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
> [INFO]    +- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile
> [INFO]    +- net.sf.kosmosfs:kfs:jar:0.3:compile
> [INFO]    +- hsqldb:hsqldb:jar:1.8.0.10:compile
> [INFO]    +- oro:oro:jar:2.0.8:compile
> [INFO]    \- org.eclipse.jdt:core:jar:3.1.1:compile
>
> I still get the same error. Is there anything specific I need to do other
> than changing dependencies in pom.xml to make this error go away?
>
> On Mon, Mar 19, 2012 at 9:12 PM, Tatu Saloranta <tsaloranta@gmail.com>wrote:
>
>> On Mon, Mar 19, 2012 at 6:06 PM, Scott Carey <scottcarey@apache.org>
>> wrote:
>> > What version of Avro are you using?
>> >
>> > You may want to try Avro 1.6.3 + Jackson 1.8.8.
>> >
>> > This is related, but is not your exact problem.
>> > https://issues.apache.org/jira/browse/AVRO-1037
>> >
>> > You are likely pulling in some other version of jackson somehow.  You
>> may
>> > want to use 'mvn dependency:tree' on your project to see where all the
>> > dependencies are coming from.  That may help identify the culprit.
>>
>> This sounds like a good idea, and I agree in that this is probably
>> still due to an old version lurking somewhere.
>>
>> -+ Tatu +-
>>
>
>
>

Mime
View raw message