incubator-kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 5)
Date Fri, 04 Nov 2011 22:35:56 GMT
> Therefore a binary distribution must include the dependent libraries
> to make it run out of the box.

I agree with Taylor here. That means our binary distribution will not
have the jars in boot/test.

>> > If someone wants to run tests they are a developer and should get a
> source distribution.

This is also a good suggestion. That means we can upload a source
distribution along with the binary distribution.

I would encourage everyone to speak up here, to avoid delaying this
release any further.

Thanks,
Neha

On Fri, Nov 4, 2011 at 9:16 AM, Taylor Gautier <tgautier@tagged.com> wrote:
> My $.02
>
> There are different audiences for different target distributions
>  - binary distribution : end user (developer) or sysadmin
>  - source distribution : a developer
>
> Therefore a binary distribution must include the dependent libraries
> to make it run out of the box.
>
> That doesn't include tests because the audience for a binary
> distribution doesn't run tests.
>
> If someone wants to run tests they are a developer and should get a
> source distribution.
>
> The source distribution should NOT contain binary dependencies. In
> this case Maven or another suitable build tool should resolve any
> dependencies during the build stage.
>
>
>
> On Nov 4, 2011, at 8:51 AM, Neha Narkhede <neha.narkhede@gmail.com> wrote:
>
>> Let me state why we included *all* the dependencies in the package
>> distribution. Initially I thought this distribution should just work
>> out-of-the-box after the download. That includes all unit tests, all
>> scripts in core as well as contrib. Note that the assumption was to not
>> have the user run ./sbt udpate to download dependencies or ./sbt package to
>> build the sub projects.
>>
>> Now, assuming we have the user do both, here is the set of jars we can
>> include -
>>
>> ./core/lib/zkclient-20110412.jar
>> ./lib/apache-rat-0.8-SNAPSHOT.jar
>> ./lib/sbt-launch.jar
>> ./contrib/hadoop-consumer/lib/avro-1.4.0.jar
>> ./contrib/hadoop-consumer/lib/commons-logging-1.0.4.jar
>> ./contrib/hadoop-consumer/lib/hadoop-0.20.2-core.jar
>> ./contrib/hadoop-consumer/lib/jackson-core-asl-1.5.5.jar
>> ./contrib/hadoop-consumer/lib/jackson-mapper-asl-1.5.5.jar
>> ./contrib/hadoop-consumer/lib/pig-0.8.0-core.jar
>> ./contrib/hadoop-consumer/lib/piggybank.jar
>> ./contrib/hadoop-producer/lib/avro-1.4.0.jar
>> ./contrib/hadoop-producer/lib/commons-logging-1.0.4.jar
>> ./contrib/hadoop-producer/lib/hadoop-0.20.2-core.jar
>> ./contrib/hadoop-producer/lib/jackson-core-asl-1.5.5.jar
>> ./contrib/hadoop-producer/lib/jackson-mapper-asl-1.5.5.jar
>> ./contrib/hadoop-producer/lib/pig-0.8.0-core.jar
>> ./contrib/hadoop-producer/lib/piggybank.jar
>> .*
>> /contrib/hadoop-consumer/target/scala_2.8.0/hadoop-consumer_2.8.0-0.7.0.jar
>> ./contrib/hadoop-producer/target/scala_2.8.0/hadoop-producer_2.8.0-0.7.0.jar
>> ./examples/target/scala_2.8.0/kafka-java-examples-0.7.0.jar
>> ./core/target/scala_2.8.0/kafka-0.7.0.jar*
>>
>> The jars in bold are Kafka jars. The question is how will the user be able
>> to run our jars, with just the stripped set of dependent jars we package ?
>>
>>>> Many of the issues about distribution would automatically be solved if
>> Maven were used.
>>
>> We use maven. All the jars in "lib_managed" are downloaded from Maven. The
>> question is not whether or not to use Maven. The question is whether you
>> have the user download dependencies build the jars themselves or not.
>>
>> Once that is clear, we can reduce the set of dependent jars we include.
>>
>> I would encourage everyone to give your inputs now, since this is important
>> to iron out for further releases.
>>
>> Thanks,
>> Neha
>>
>> On Fri, Nov 4, 2011 at 8:26 AM, Alan D. Cabrera <list@toolazydogs.com>
>> wrote:
>>> It would...
>>>
>>> Many of the issues about distribution would automatically be solved if
>> Maven were used.
>>>
>>> <disclaimer>I'm a Maven zealot</disclaimer>
>>>
>>> On Nov 4, 2011, at 8:17 AM, Mark wrote:
>>>
>>>> In regards to the size of the distribution, wouldn't a mavenized build
>> help with this?
>>>>
>>>> On 11/4/11 7:42 AM, Alan D. Cabrera wrote:
>>>>> On Nov 4, 2011, at 7:29 AM, Chris Burroughs wrote:
>>>>>
>>>>>> On 11/03/2011 03:41 PM, Alan D. Cabrera wrote:
>>>>>>> Zowie.  Look at all these jars that I have to make sure are
kosher to
>> distribute:
>>>>>>>
>>>>>>>
>>>>>>> Can someone help me?
>>>>>>>
>>>>>> Assuming we can cut down on the boot/test jars. All we need is a
table
>>>>>> of "jar-name,licence", correct?
>>>>> As far as the number of jars I am only concerned with regard to the
>> size of the distribution, 50M.  That seems excessive to me and provides no
>> real value given that the consumers of the distribution can easily build
>> the product themselves.
>>>>>
>>>>> With that said, yes, it would be helpful of there was a list:
>>>>>
>>>>> jar name, license, URL that indicates the license for the jar
>>>>>
>>>>>
>>>>> Regards,
>>>>> Alan
>>>>>
>>>
>>>
>

Mime
View raw message