incubator-bigtop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <>
Subject Re: when dependencies in common to more than one project appear on the classpath
Date Fri, 08 Jun 2012 22:29:40 GMT
+David (with whom we just spoke about this very subject):

On Fri, Jun 8, 2012 at 2:54 PM, Andrew Purtell <> wrote:
> This is a HBase shell nit specifically, and the warning from SLF4J is
> purely a warning:
>    SLF4J: Class path contains multiple SLF4J bindings.
>    SLF4J: Found binding in
> [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>    SLF4J: Found binding in
> [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>    SLF4J: See for
> an explanation.
> but there are other cases where dependencies in common to more than
> one project will appear on the classpath, sometimes with differing
> versions. Jackson comes to mind.

This is very much a problem for Bigtop (and frankly any kind of Hadoop
distribution on the market) although it hasn't caused us much trouble
yet. Still, I really think we should address this. Here's a JIRA that
tracks the effort:
this is clearly post Bigtop 0.4.0 but it could very well make it into
Bigtop 0.5.0.

Of course, this is half of the problem -- the other half is making sure
that we can harmonize the versions of all the dependencies between
the projects. We do that in CDH (via patching poms, etc) but haven't
done anything like that in Bigtop yet.

> How this would be handled at the OS level is each common dependency
> would be factored out into a library package.

I'm keeping an eye on what Linux vendors do to address this issue. Here's
a blog post that is well worth a read. I think whatever we end up implementing
in Bigtop needs to be aligned (to the best possible extent) with Linux vendors:

Of course, if anybody has any concrete proposal I'd love for them to
be articulated on the JIRA or mailing list -- do let me know!


View raw message