spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tobias Pfeiffer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-2492) KafkaReceiver minor changes to align with Kafka 0.8
Date Thu, 24 Jul 2014 03:52:39 GMT

    [ https://issues.apache.org/jira/browse/SPARK-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072759#comment-14072759
] 

Tobias Pfeiffer commented on SPARK-2492:
----------------------------------------

Hi,

thanks for updating the pull request! The code looks good to me! (I haven't tested it, though.)
My only suggestion would be to change the comment

{quote}
To avoid concurrent deleting Zookeeper metadata in each Receiver when multiple consumers are
launched, this should be call be createStream().
{quote}
to
{quote}
To avoid deleting existing Zookeeper metadata when multiple consumers are launched,
this should be called before createStream().
{quote}

Maybe someone with write permissions can review as well and merge it?

Thanks!

> KafkaReceiver minor changes to align with Kafka 0.8 
> ----------------------------------------------------
>
>                 Key: SPARK-2492
>                 URL: https://issues.apache.org/jira/browse/SPARK-2492
>             Project: Spark
>          Issue Type: Improvement
>          Components: Streaming
>    Affects Versions: 1.0.0
>            Reporter: Saisai Shao
>            Assignee: Saisai Shao
>            Priority: Minor
>             Fix For: 1.1.0
>
>
> Update to delete Zookeeper metadata when Kafka's parameter "auto.offset.reset" is set
to "smallest", which is aligned with Kafka 0.8's ConsoleConsumer.
> Also use Kafka offered API without directly using zkClient.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message