hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: hadoop JARs not in lib/ directory of layout
Date Thu, 04 Aug 2011 22:47:43 GMT
It is easier to write shell script to import jar files by directory instead of explicitly reference
to a few jars with specific versions.

The common use case is:

HBase needs to use hadoop jar files, but HBase depends on more recent version of log4j.  The
construction of the class path would be:


This provides a way to segment the library loading with least amount of scripting and loosely


On Aug 4, 2011, at 2:40 PM, Alejandro Abdelnur wrote:

> [moving to core-dev@, general@ BCCed]
> Eric,
> Even if the JAR is in lib/ you could import/use that JAR only.
> How would you use Hadoop JARs without its dependencies? Many things will
> break unless you add the dependency JARs.
> Granted, there are JARs that are used by Hadoop server side only
> (JT/NN/TT/DN/SNN), but that is a different thing. Having a client side set
> of JARs would help handle this (MAPREDUCE-1638).
> Thoughts?
> Thanks.
> Alejandro
> On Thu, Aug 4, 2011 at 1:14 PM, Eric Yang <eric818@gmail.com> wrote:
>> The jar files placement  outside of lib directory is to ensure the project
>> generated jar files are not mixed with it's dependencies.
>> Hence, if another project tries to import current project's jar files
>> without dependencies, it is possible to do so.
>> regards,
>> Eric
>> On Aug 4, 2011, at 11:03 AM, Alejandro Abdelnur wrote:
>>> What is the rationale for having the hadoop JARs outside of the lib/
>>> directory?
>>> It would definitely simplify packaging configuration if they are under
>> lib/
>>> as well.
>>> Any objection to it?
>>> Thanks.
>>> Alejandro

View raw message