flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4822) Ensure that the Kafka 0.8 connector is compatible with kafka-consumer-groups.sh
Date Wed, 22 Nov 2017 11:21:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16262328#comment-16262328

ASF GitHub Bot commented on FLINK-4822:

GitHub user taizilongxu opened a pull request:


    [FLINK-4822] Ensure that the Kafka 0.8 connector is compatible with k…

    ## What is the purpose of the change
    When we deploy the taskmanager in docker of our cluster, it's hard to locate which taskmanager
cosnume the right partition of kafka except looking up the  log in docker, so I just add the
owner in zk path when PeriodOffsetCommitter the offset.
    ## Brief change log
      - add the registerPartitionOwnership  when  commit the offset to zookeeper, and store
the info like :  /consumers/[group_id]/owner/[topic]/[partition_id] 
    ## Verifying this change
    This change is a trivial rework / code cleanup without any test coverage.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency):  no
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`:no
      - The serializers:  no 
      - The runtime per-record code paths (performance sensitive): no
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: no
      - The S3 file system connector: no
    ## Documentation
      - Does this pull request introduce a new feature? no
      - If yes, how is the feature documented? not documented

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/taizilongxu/flink flink-4822

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5050
commit 1c7a345e671010cf07b02cfdff9968ced38e2632
Author: xuxiao.xu <xuxiao.xu@qunar.com>
Date:   2017-11-22T09:36:12Z

    [FLINK-4822] Ensure that the Kafka 0.8 connector is compatible with kafka-consumer-groups.sh


> Ensure that the Kafka 0.8 connector is compatible with kafka-consumer-groups.sh
> -------------------------------------------------------------------------------
>                 Key: FLINK-4822
>                 URL: https://issues.apache.org/jira/browse/FLINK-4822
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>            Reporter: Robert Metzger
> The Kafka 0.8 connector is not properly creating all datastructures in Zookeeper for
Kafka's {{kafka-consumer-groups.sh}} tool.
> A user reported the issue here: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-connector08-not-updating-the-offsets-with-the-zookeeper-td9469.html#a9498
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

This message was sent by Atlassian JIRA

View raw message