hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject hadoop jar question
Date Sat, 03 Nov 2012 23:14:52 GMT
i am looking at the code for RunJar.java which is behind "hadoop jar" for
hadoop 0.20.2 (from cdh3u5).

i see
1) jar is unpacked to a  temporary directory
2) the file URLs of all the jars found in the lib subdir of the unpacked
jar are gathered into a list called classPath
3) a new ClassLoader loader is created of type URLClassLoader using the
list of URLs classPath from 2)
4) the classloader for the current thread is set to newly created loader
5) the main method of the user provided class is launched using the new

given all this, i would say the new loader will be consulted first
(locally, not on cluster) when my classes need to be loaded, which means
classes found in the jars inside the lib folder of the provided jar should
get preference. yet what i observe is that the classes from ancient jackson
jar that is included with hadoop get preference over the ones i includ (by
putting newer jackson jars inside the lib folder of my jar). why is this?
what am i missing?


View raw message