kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Carey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-865) Mavenize and separate the client.
Date Thu, 18 Apr 2013 18:26:14 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635478#comment-13635478

Scott Carey commented on KAFKA-865:

I'm a big fan of Maven (It has some big flaws, but so does SBT and ant), but it is not necessary
for this.  SBT can publish artifacts to maven central with proper pom files.

I'm not sure how much the client dependency tree differs from the server one at the moment
-- perhaps only with zookeeper.  I do feel that it is important to have dependencies declared
in the published pom be accurate and not contain cruft for different use cases (e.g. producer
vs consumer may have different transitive dependencies, and thus should have different artifacts).

> Mavenize and separate the client.
> ---------------------------------
>                 Key: KAFKA-865
>                 URL: https://issues.apache.org/jira/browse/KAFKA-865
>             Project: Kafka
>          Issue Type: Improvement
>          Components: clients
>    Affects Versions: 0.8
>            Reporter: Ashwanth Fernando
> It seems that the java client for Kafka is also bundled with the server JAR file and
this is generated using sbt package. This is difficult for java folks to work with because:
> 1) Many java shops use maven (and a lot of them have a Sonatype Nexus repository in house)
for dependency management. They want to specify the GAV and bang, the client jar and all its
dependencies should be added to the application's classpath. I can't do that right now, because
I need to run ./sbt eclipse, get the .JAR, add that to my classpath, add a whole lot of dependencies
(log4j, slf4j, zkClient and so on) manually, which is a pain. 
> There are 90 million maven central uploads/downloads in 2012 alone. Almost all the java
shops out there have maven (either central or in house sonatype).
> 2) Separation of concerns - keeping the server (core) and the client's classes increases
the size of the bundle for the client and also everytime the server's code changes and a release
is performed, the client also needs to update their .JAR file. which is not very great. We
don't want a ton of clients to update their .JAR file, just because a faster replication strategy
for my kafka cluster changed in a new release.
> Action items are to separate the client portion of Kafka, add it in a pom along with
the compile time dependencies and upload it to Maven Central or if you have a LinkedIn externally
exposed Nexus, over there.
> This will increase adoption of the Kafka framework.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message