storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Spico Florin <>
Subject Big fat jar
Date Thu, 04 Feb 2016 11:50:38 GMT
  After building my project that comtaint the topology, I have a big fat
jar of 75MB. I have dependencies on HBase, OpenTSDB and Kafka. I would like
to reduce the size of the the jar due to the fact that we can a lot of
instances of the topology running (aprox 100).

I have read in two posts:

I found two solutions.
One is to put the commons libararies in Storm_install_dir/lib   and the
second one is to put them in folder USER_CONF_DIR defined in storm.yaml. In
both acases we should take care to keep a single version of dependent
libraries. This could be a potential issue, due to the fact that their
might be common libraries that can interfere with storm libraries (such as
zookeeper or appache-commons).
 Also, I'm running Storm from Ambari.

In the above scenario:
- what could be the best solution to keep the common dependent libraries?
- If I configure the USER_CONF_DIR in Ambari will it be propagated to all
cluster machines?

I look forward for your answers.
Thanks .

View raw message