spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reynold Xin (JIRA)" <>
Subject [jira] [Commented] (SPARK-13877) Consider removing Kafka modules from Spark / Spark Streaming
Date Thu, 17 Mar 2016 18:17:33 GMT


Reynold Xin commented on SPARK-13877:

I'm not 100% sure it is a good idea to move it out (it might be), but I'm definitely against
renaming the package name.

Given Kafka is one of the most widely used streaming sources in 1.x, moving this out and changing
the package name means we are breaking almost every spark streaming app out there when they
upgrade to 2.x. It seems like we are doing all these extra work just to break things.

> Consider removing Kafka modules from Spark / Spark Streaming
> ------------------------------------------------------------
>                 Key: SPARK-13877
>                 URL:
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core, Streaming
>    Affects Versions: 1.6.1
>            Reporter: Hari Shreedharan
> Based on the discussion the PR for SPARK-13843 ([here|]),
we should consider moving the Kafka modules out of Spark as well. 
> Providing newer functionality (like security) has become painful while maintaining compatibility
with older versions of Kafka. Moving this out allows more flexibility, allowing users to mix
and match Kafka and Spark versions.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message