kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Douglas <cdoug...@apache.org>
Subject Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 5)
Date Sun, 06 Nov 2011 05:32:30 GMT
Sorry, I didn't see this. I'm subscribed to kafka-dev, but not kafka-users.

As Alan has often stated, issues like these are judgement calls. Kafka
may bundle ASL-compatible jars with its release tarball. Other
projects, like Hadoop, have done this in the past. I have almost no
experience distributing or consuming Scala projects, so I can't say
anything about downstream users' expectations. Speaking generally:

In its favor, the "out of the box" experience is simpler,
configuration is trivial, one can use the software without net access,
and FAQs on handling common quirks in a user's environment are shorter
for it. Against it, the distribution is considerably larger than it
needs to be, projects depending on this one may endure more
integration hassles, and the licensing for redistribution (auditing
each jar, ensuring all attribution clauses are satisfied, etc.) is
avoidable pain. The ASF is principally about producing source code,
but if the project wants to generate and distribute a tarball with
binary artifacts then I know of no policy preventing it.

I went through the included jars and didn't see anything that couldn't
be redistributed. I also found all the licenses (included with the
jars) that were required. I think the current release can go out
as-is. I prefer source distributions and publishing to maven, but
(again) I don't know others' downstream expectations. -C

On Sat, Nov 5, 2011 at 12:04 AM, Neha Narkhede <neha.narkhede@gmail.com> wrote:
> Alan/Chris,
>
> Please can you take a look at the alternatives here and let us know
> your thoughts ?
>
> Thanks,
> Neha
>
>
> ---------- Forwarded message ----------
> From: Neha Narkhede <neha.narkhede@gmail.com>
> Date: Fri, Nov 4, 2011 at 4:20 PM
> Subject: Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 5)
> To: kafka-users@incubator.apache.org
>
>
> Alan/Chris,
>
> As the mentors/champion, I would encourage you to chime in here. We
> really want to make progress on this release. The issue found
> yesterday actually existed since the first RC. It will help to iron
> out issues all at once, instead of one RC at a time.
>
> Thanks,
> Neha
>
> On Fri, Nov 4, 2011 at 4:16 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
>> Fully agree. I don't particularly care if we have two distribution files or
>> one, but I think it is a bad user experience for the person who just wants
>> to start using our software to have to use SBT to fetch the jar files or
>> worse yet build it from source. Relying on maven to download jars doesn't
>> actually save anyone download time, it just adds more steps (you eventually
>> need the jars).
>>
>> -Jay
>>
>> On Fri, Nov 4, 2011 at 3:35 PM, Neha Narkhede <neha.narkhede@gmail.com>wrote:
>>
>>> > Therefore a binary distribution must include the dependent libraries
>>> > to make it run out of the box.
>>>
>>> I agree with Taylor here. That means our binary distribution will not
>>> have the jars in boot/test.
>>>
>>> >> > If someone wants to run tests they are a developer and should get
a
>>> > source distribution.
>>>
>>> This is also a good suggestion. That means we can upload a source
>>> distribution along with the binary distribution.
>>>
>>> I would encourage everyone to speak up here, to avoid delaying this
>>> release any further.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Fri, Nov 4, 2011 at 9:16 AM, Taylor Gautier <tgautier@tagged.com>
>>> wrote:
>>> > My $.02
>>> >
>>> > There are different audiences for different target distributions
>>> >  - binary distribution : end user (developer) or sysadmin
>>> >  - source distribution : a developer
>>> >
>>> > Therefore a binary distribution must include the dependent libraries
>>> > to make it run out of the box.
>>> >
>>> > That doesn't include tests because the audience for a binary
>>> > distribution doesn't run tests.
>>> >
>>> > If someone wants to run tests they are a developer and should get a
>>> > source distribution.
>>> >
>>> > The source distribution should NOT contain binary dependencies. In
>>> > this case Maven or another suitable build tool should resolve any
>>> > dependencies during the build stage.
>>> >
>>> >
>>> >
>>> > On Nov 4, 2011, at 8:51 AM, Neha Narkhede <neha.narkhede@gmail.com>
>>> wrote:
>>> >
>>> >> Let me state why we included *all* the dependencies in the package
>>> >> distribution. Initially I thought this distribution should just work
>>> >> out-of-the-box after the download. That includes all unit tests, all
>>> >> scripts in core as well as contrib. Note that the assumption was to
not
>>> >> have the user run ./sbt udpate to download dependencies or ./sbt
>>> package to
>>> >> build the sub projects.
>>> >>
>>> >> Now, assuming we have the user do both, here is the set of jars we can
>>> >> include -
>>> >>
>>> >> ./core/lib/zkclient-20110412.jar
>>> >> ./lib/apache-rat-0.8-SNAPSHOT.jar
>>> >> ./lib/sbt-launch.jar
>>> >> ./contrib/hadoop-consumer/lib/avro-1.4.0.jar
>>> >> ./contrib/hadoop-consumer/lib/commons-logging-1.0.4.jar
>>> >> ./contrib/hadoop-consumer/lib/hadoop-0.20.2-core.jar
>>> >> ./contrib/hadoop-consumer/lib/jackson-core-asl-1.5.5.jar
>>> >> ./contrib/hadoop-consumer/lib/jackson-mapper-asl-1.5.5.jar
>>> >> ./contrib/hadoop-consumer/lib/pig-0.8.0-core.jar
>>> >> ./contrib/hadoop-consumer/lib/piggybank.jar
>>> >> ./contrib/hadoop-producer/lib/avro-1.4.0.jar
>>> >> ./contrib/hadoop-producer/lib/commons-logging-1.0.4.jar
>>> >> ./contrib/hadoop-producer/lib/hadoop-0.20.2-core.jar
>>> >> ./contrib/hadoop-producer/lib/jackson-core-asl-1.5.5.jar
>>> >> ./contrib/hadoop-producer/lib/jackson-mapper-asl-1.5.5.jar
>>> >> ./contrib/hadoop-producer/lib/pig-0.8.0-core.jar
>>> >> ./contrib/hadoop-producer/lib/piggybank.jar
>>> >> .*
>>> >>
>>> /contrib/hadoop-consumer/target/scala_2.8.0/hadoop-consumer_2.8.0-0.7.0.jar
>>> >>
>>> ./contrib/hadoop-producer/target/scala_2.8.0/hadoop-producer_2.8.0-0.7.0.jar
>>> >> ./examples/target/scala_2.8.0/kafka-java-examples-0.7.0.jar
>>> >> ./core/target/scala_2.8.0/kafka-0.7.0.jar*
>>> >>
>>> >> The jars in bold are Kafka jars. The question is how will the user be
>>> able
>>> >> to run our jars, with just the stripped set of dependent jars we
>>> package ?
>>> >>
>>> >>>> Many of the issues about distribution would automatically be
solved if
>>> >> Maven were used.
>>> >>
>>> >> We use maven. All the jars in "lib_managed" are downloaded from Maven.
>>> The
>>> >> question is not whether or not to use Maven. The question is whether
you
>>> >> have the user download dependencies build the jars themselves or not.
>>> >>
>>> >> Once that is clear, we can reduce the set of dependent jars we include.
>>> >>
>>> >> I would encourage everyone to give your inputs now, since this is
>>> important
>>> >> to iron out for further releases.
>>> >>
>>> >> Thanks,
>>> >> Neha
>>> >>
>>> >> On Fri, Nov 4, 2011 at 8:26 AM, Alan D. Cabrera <list@toolazydogs.com>
>>> >> wrote:
>>> >>> It would...
>>> >>>
>>> >>> Many of the issues about distribution would automatically be solved
if
>>> >> Maven were used.
>>> >>>
>>> >>> <disclaimer>I'm a Maven zealot</disclaimer>
>>> >>>
>>> >>> On Nov 4, 2011, at 8:17 AM, Mark wrote:
>>> >>>
>>> >>>> In regards to the size of the distribution, wouldn't a mavenized
build
>>> >> help with this?
>>> >>>>
>>> >>>> On 11/4/11 7:42 AM, Alan D. Cabrera wrote:
>>> >>>>> On Nov 4, 2011, at 7:29 AM, Chris Burroughs wrote:
>>> >>>>>
>>> >>>>>> On 11/03/2011 03:41 PM, Alan D. Cabrera wrote:
>>> >>>>>>> Zowie.  Look at all these jars that I have to make
sure are kosher
>>> to
>>> >> distribute:
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Can someone help me?
>>> >>>>>>>
>>> >>>>>> Assuming we can cut down on the boot/test jars. All
we need is a
>>> table
>>> >>>>>> of "jar-name,licence", correct?
>>> >>>>> As far as the number of jars I am only concerned with regard
to the
>>> >> size of the distribution, 50M.  That seems excessive to me and provides
>>> no
>>> >> real value given that the consumers of the distribution can easily build
>>> >> the product themselves.
>>> >>>>>
>>> >>>>> With that said, yes, it would be helpful of there was a
list:
>>> >>>>>
>>> >>>>> jar name, license, URL that indicates the license for the
jar
>>> >>>>>
>>> >>>>>
>>> >>>>> Regards,
>>> >>>>> Alan
>>> >>>>>
>>> >>>
>>> >>>
>>> >
>>>
>>
>

Mime
View raw message