hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Armstrong <john.armstr...@ccri.com>
Subject Overriding remote classes
Date Wed, 14 Dec 2011 13:20:55 GMT
Hi, there.

I've run into an odd situation, and I'm wondering if there's a way around
it; I'm trying to use Jackson for some JSON serialization in my program,
and I wrote/unit-tested it to work with Jackson 1.9.  Then, in integration
testing, I started to see some weird version incompatibilities and
AbstractMethodErrors.  Indeed, some digging revealed that our Hadoop
installation (CDH3b3, incidentally) has the Jackson 1.5.2 JARs in its
$HADOOP_HOME/lib directory which, as I understand it, forms the basis of
the remote JVM classpath.

So, for now I've rewritten our code to use the 1.5.2 libraries, but it's
ugly and hacky in some places due to Jackson 1.5.2 not having a sensible
TypeFactory or anything like that.  I'm wondering, though, if there's a way
to make the remove JVM use *our* versions of the Jackson libraries
(packaged in the fat JAR) instead of the ones that come with Hadoop.

And no, in deployment we will not be able to control the cluster ourselves
and rip out the old JARs or replace them with updated ones.

View raw message