avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <scottca...@apache.org>
Subject Re: Java MapReduce Avro Jackson Error
Date Tue, 27 Mar 2012 03:25:53 GMT
What happens if you remove avro-tools entirely?

What in there are you using?  It is all command-line tools, if there is
anything in there you need, you can use the equivalent Java API instead.  If
there is something in there of use that is not a command line tool, we
should make it available in the other modules.

What is the stack trace in full, not just the top error?  What code (Avro or
Hadoop) is triggering it?

"Error: 
org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Fea
ture;)Lorg/codehaus/jackson/JsonFactory;"

is not enough info if the basic things are not working.

On 3/26/12 7:58 PM, "Deepak Nettem" <deepaknettem@gmail.com> wrote:

> Hi Folks,
> 
> This issue is still not resolved :( Any other ideas?
> 
> Best,
> Deepak
> 
> On Mon, Mar 19, 2012 at 11:25 PM, Scott Carey <scottcarey@apache.org> wrote:
>> If you are using avro-tools, beware it is a shaded jar with all dependencies
>> inside of it for use as a command line tool (java ­jar
>> avro-tools-VERSION.jar).
>> 
>> If you are using avro-tools in your project for some reason (there is really
>> only command line utilities in it) use the nodeps classifier:
>> 
>> <classifier>nodeps</classifier>
>> 
>> http://repo1.maven.org/maven2/org/apache/avro/avro-tools/1.6.3/
>> 
>> Note the nodeps jar is 47K, while the default jar is 10MB.
>> 
>> 
>> For what it is worth, I removed the Jackson jar from our hadoop install long
>> ago.  It is used to dump configuration files to JSON there, a peripheral
>> feature we don't use.
>> 
>> Another thing that you may want to do is change your Hadoop dependency scope
>> to
>> <scope>provided</scope> since hadoop will be put on your classpath by
the
>> hadoop environment.   Short of this, excluding the chained Hadoop
>> dependencies you aren't using (most likely: jetty,  kfs, and the
>> tomcat:jasper and eclipse:jdt stuff) may help.
>> 
>> On 3/19/12 6:23 PM, "Deepak Nettem" <deepaknettem@gmail.com> wrote:
>> 
>>> Hi Tatu / Scott,
>>> 
>>> Thanks for your replies. I replaced the earlier dependencies with these:
>>> 
>>>    <dependency>
>>>     <groupId>org.apache.avro</groupId>
>>>     <artifactId>avro-tools</artifactId>
>>>     <version>1.6.3</version>
>>>     </dependency>
>>>     
>>>     <dependency>
>>>     <groupId>org.apache.avro</groupId>
>>>     <artifactId>avro</artifactId>
>>>     <version>1.6.3</version>
>>>     </dependency>
>>> 
>>>     <dependency>
>>>     <groupId>org.codehaus.jackson</groupId>
>>>       <artifactId>jackson-mapper-asl</artifactId>
>>>       <version>1.8.8</version>
>>>       <scope>compile</scope>
>>>     </dependency>
>>> 
>>>     <dependency>
>>>     <groupId>org.codehaus.jackson</groupId>
>>>       <artifactId>jackson-core-asl</artifactId>
>>>       <version>1.8.8</version>
>>>       <scope>compile</scope>
>>>     </dependency>
>>> 
>>> And this is my app dependency tree:
>>> 
>>> [INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ AvroTest ---
>>> [INFO] org.avrotest:AvroTest:jar:1.0-SNAPSHOT
>>> [INFO] +- junit:junit:jar:3.8.1:test (scope not updated to compile)
>>> [INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
>>> [INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
>>> [INFO] +- net.sf.json-lib:json-lib:jar:jdk15:2.3:compile
>>> [INFO] |  +- commons-beanutils:commons-beanutils:jar:1.8.0:compile
>>> [INFO] |  +- commons-collections:commons-collections:jar:3.2.1:compile
>>> [INFO] |  +- commons-lang:commons-lang:jar:2.4:compile
>>> [INFO] |  +- commons-logging:commons-logging:jar:1.1.1:compile
>>> [INFO] |  \- net.sf.ezmorph:ezmorph:jar:1.0.6:compile
>>> [INFO] +- org.apache.avro:avro-tools:jar:1.6.3:compile
>>> [INFO] |  \- org.slf4j:slf4j-api:jar:1.6.4:compile
>>> [INFO] +- org.apache.avro:avro:jar:1.6.3:compile
>>> [INFO] |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
>>> [INFO] |  \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
>>> [INFO] \- org.apache.hadoop:hadoop-core:jar:0.20.2:compile
>>> [INFO]    +- commons-cli:commons-cli:jar:1.2:compile
>>> [INFO]    +- xmlenc:xmlenc:jar:0.52:compile
>>> [INFO]    +- commons-httpclient:commons-httpclient:jar:3.0.1:compile
>>> [INFO]    +- commons-codec:commons-codec:jar:1.3:compile
>>> [INFO]    +- commons-net:commons-net:jar:1.4.1:compile
>>> [INFO]    +- org.mortbay.jetty:jetty:jar:6.1.14:compile
>>> [INFO]    +- org.mortbay.jetty:jetty-util:jar:6.1.14:compile
>>> [INFO]    +- tomcat:jasper-runtime:jar:5.5.12:compile
>>> [INFO]    +- tomcat:jasper-compiler:jar:5.5.12:compile
>>> [INFO]    +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile
>>> [INFO]    +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile
>>> [INFO]    |  \- ant:ant:jar:1.6.5:compile
>>> [INFO]    +- commons-el:commons-el:jar:1.0:compile
>>> [INFO]    +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile
>>> [INFO]    +- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile
>>> [INFO]    +- net.sf.kosmosfs:kfs:jar:0.3:compile
>>> [INFO]    +- hsqldb:hsqldb:jar:1.8.0.10:compile
>>> [INFO]    +- oro:oro:jar:2.0.8:compile
>>> [INFO]    \- org.eclipse.jdt:core:jar:3.1.1:compile
>>> 
>>> I still get the same error. Is there anything specific I need to do other
>>> than changing dependencies in pom.xml to make this error go away?
>>> 
>>> On Mon, Mar 19, 2012 at 9:12 PM, Tatu Saloranta <tsaloranta@gmail.com>
>>> wrote:
>>>> On Mon, Mar 19, 2012 at 6:06 PM, Scott Carey <scottcarey@apache.org>
wrote:
>>>>> > What version of Avro are you using?
>>>>> >
>>>>> > You may want to try Avro 1.6.3 + Jackson 1.8.8.
>>>>> >
>>>>> > This is related, but is not your exact problem.
>>>>> > https://issues.apache.org/jira/browse/AVRO-1037
>>>>> >
>>>>> > You are likely pulling in some other version of jackson somehow.
 You
>>>>> may
>>>>> > want to use 'mvn dependency:tree' on your project to see where all
the
>>>>> > dependencies are coming from.  That may help identify the culprit.
>>>> 
>>>> This sounds like a good idea, and I agree in that this is probably
>>>> still due to an old version lurking somewhere.
>>>> 
>>>> -+ Tatu +-
>>> 
>>> 
> 
> 



Mime
View raw message