On = Feb 6, 2014, at 9:28 PM, Adam Lewis <mail@adamlewis.com> = wrote:

My $0.02 on this = subject:

Without going down the path of class loader or OSGi mania and becoming a = full container, I'd definitely be in favor of storm relocating its own = dependencies. In this way edge cases around things like reflection = can be handled once within storm rather than burdening every topology = builder with those details. Part of the problem seems to be that = storm makes extensive use (directly or transitively) of a lot of go-to = utility libraries like guava, thrift, jodatime, json-simple, snakeyaml, = commons-io, etc... I'm sure that leveraging these libraries allowed = storm's development to proceed rapidly, but from a maturity perspective, = it is problematic to impose these version choices on users. And = while I might want Storm to, say, try to track the latest Guava version, = that same policy could be very problematic for others.

If storm can relocate = even some of its own dependencies, I think that would be a great help to = me at least. Longer term, I wonder how much of some of these = libraries are really being used. For example, is clj-time (and by = extension joda-time) really needed? Or just a small fraction of the = functionality in that library? I can probably pitch in some of the = effort required to do this, if this is the direction people want to go = in.

On Thu, Feb 6, 2014 at 8:44 PM, P. Taylor Goetz <ptgoetz@gmail.com> wrote:

I=92m glad the shader plugin worked = for you.

Updating dependencies can be tricky as = it can easily introduce = regressions.

Ultimately we need to figure = out the best solution to avoiding conflicts between user code (i.e. = dependencies in topology jar files) and Storm=92s libraries.

The classloader approach has been attempted, but IMO = Storm=92s use of serialization complicates things significantly. Package = relocation seems to be a relatively lightweight = solution.

If that=92s a direction we pursue, then it introduces the question = of whether Storm should relocate its dependencies, or if that should be = left up to the user (topology = developer).

Elastic Search has gone down the = path of relocating some of their dependencies [1] (not necessarily an = endorsement, just an observation).

I=92ve CC=92d dev@ since this is all related to the = infamous issue #115, which is now STORM-129 = [2].

- = Taylor

[1] https://github.com/elasticsearch/elasticsearch/blob/mast= er/pom.xml#L474

[2] https://issues.apache.org/jira/browse/STORM-129

On Feb 6, 2014, at 7:25 PM, Vinay Pothnis <vinay.pothnis@gmail.com> = wrote:

Thank you all = for replies! The shader-plugin solution seems to work for us.

I wonder if we can create a JIRA ticket for storm to = upgrade the http-client library as part of their next release.

-Vinay

On Thu, Feb 6, = 2014 at 2:38 PM, Michael Rose <michael@fullcontact.com> wrote:

We've = done this with SLF4j and Guava as well without issues.

Michael Rose (@Xorlev)
Senior Platform = Engineer, FullContact
michael@fullcontact.com

On Thu, Feb 6, 2014 at 3:03 PM, Mark = Greene <mark@evertrue.com> wrote:

We had this problem as well. We modified our chef = cookbook to just replace the older version with the newer one and storm = didn't complain or have any other issues as a result.

On Wed, Feb 5, 2014 at 10:31 AM, P. = Taylor Goetz <ptgoetz@gmail.com> wrote:
Your best bet is probably to use the shade plugin to relocate the = http-client package so it doesn=92t conflict with the version storm = uses.

Storm does this with the libtrhift dependency in storm-core:

https://github.com/apache/incubator-storm/blob/master/st= orm-core/pom.xml#L220

(You can ignore the clojure transformer in that config, unless you have = non-AOT clojure code that uses the http-client library).

More information on using the shade plugin to do package relocations can = be found here:

http://maven.apache.org/plugins/maven-shade-plugin/examp= les/class-relocation.html

- Taylor

On Feb 4, 2014, at 4:27 PM, Vinay Pothnis <vinay.pothnis@gmail.com> wrote:

> Hello,
>
> I am using storm version 0.9.0.1.
> My application depends on apache http-client version 4.3.2 - but = storm depends on http-client version 4.1.1.
>
> What is the best way to override this dependency?
>
> Thanks
> Vinay

=