kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 5)
Date Fri, 04 Nov 2011 15:51:20 GMT
Let me state why we included *all* the dependencies in the package
distribution. Initially I thought this distribution should just work
out-of-the-box after the download. That includes all unit tests, all
scripts in core as well as contrib. Note that the assumption was to not
have the user run ./sbt udpate to download dependencies or ./sbt package to
build the sub projects.

Now, assuming we have the user do both, here is the set of jars we can
include -

./core/lib/zkclient-20110412.jar
./lib/apache-rat-0.8-SNAPSHOT.jar
./lib/sbt-launch.jar
./contrib/hadoop-consumer/lib/avro-1.4.0.jar
./contrib/hadoop-consumer/lib/commons-logging-1.0.4.jar
./contrib/hadoop-consumer/lib/hadoop-0.20.2-core.jar
./contrib/hadoop-consumer/lib/jackson-core-asl-1.5.5.jar
./contrib/hadoop-consumer/lib/jackson-mapper-asl-1.5.5.jar
./contrib/hadoop-consumer/lib/pig-0.8.0-core.jar
./contrib/hadoop-consumer/lib/piggybank.jar
./contrib/hadoop-producer/lib/avro-1.4.0.jar
./contrib/hadoop-producer/lib/commons-logging-1.0.4.jar
./contrib/hadoop-producer/lib/hadoop-0.20.2-core.jar
./contrib/hadoop-producer/lib/jackson-core-asl-1.5.5.jar
./contrib/hadoop-producer/lib/jackson-mapper-asl-1.5.5.jar
./contrib/hadoop-producer/lib/pig-0.8.0-core.jar
./contrib/hadoop-producer/lib/piggybank.jar
.*
/contrib/hadoop-consumer/target/scala_2.8.0/hadoop-consumer_2.8.0-0.7.0.jar
./contrib/hadoop-producer/target/scala_2.8.0/hadoop-producer_2.8.0-0.7.0.jar
./examples/target/scala_2.8.0/kafka-java-examples-0.7.0.jar
./core/target/scala_2.8.0/kafka-0.7.0.jar*

The jars in bold are Kafka jars. The question is how will the user be able
to run our jars, with just the stripped set of dependent jars we package ?

>> Many of the issues about distribution would automatically be solved if
Maven were used.

We use maven. All the jars in "lib_managed" are downloaded from Maven. The
question is not whether or not to use Maven. The question is whether you
have the user download dependencies build the jars themselves or not.

Once that is clear, we can reduce the set of dependent jars we include.

I would encourage everyone to give your inputs now, since this is important
to iron out for further releases.

Thanks,
Neha

On Fri, Nov 4, 2011 at 8:26 AM, Alan D. Cabrera <list@toolazydogs.com>
wrote:
> It would...
>
> Many of the issues about distribution would automatically be solved if
Maven were used.
>
> <disclaimer>I'm a Maven zealot</disclaimer>
>
> On Nov 4, 2011, at 8:17 AM, Mark wrote:
>
>> In regards to the size of the distribution, wouldn't a mavenized build
help with this?
>>
>> On 11/4/11 7:42 AM, Alan D. Cabrera wrote:
>>> On Nov 4, 2011, at 7:29 AM, Chris Burroughs wrote:
>>>
>>>> On 11/03/2011 03:41 PM, Alan D. Cabrera wrote:
>>>>> Zowie.  Look at all these jars that I have to make sure are kosher to
distribute:
>>>>>
>>>>>
>>>>> Can someone help me?
>>>>>
>>>> Assuming we can cut down on the boot/test jars. All we need is a table
>>>> of "jar-name,licence", correct?
>>> As far as the number of jars I am only concerned with regard to the
size of the distribution, 50M.  That seems excessive to me and provides no
real value given that the consumers of the distribution can easily build
the product themselves.
>>>
>>> With that said, yes, it would be helpful of there was a list:
>>>
>>> jar name, license, URL that indicates the license for the jar
>>>
>>>
>>> Regards,
>>> Alan
>>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message