kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From guozhangwang <...@git.apache.org>
Subject [GitHub] kafka pull request #2012: [WIP] KAFKA-4117: Stream partitionassignro cleanup
Date Tue, 11 Oct 2016 15:48:54 GMT
GitHub user guozhangwang opened a pull request:

    https://github.com/apache/kafka/pull/2012

    [WIP] KAFKA-4117: Stream partitionassignro cleanup

    1. Create a new ClientMetadata` to collapse `Set<String> consumerMemberIds`, `ClientState<TaskId>
state`, and `HostInfo hostInfo`.
    
    2. Stop reusing `stateChangelogTopicToTaskIds` and `internalSourceTopicToTaskIds` to access
the (sub-)topology's internal repartition and changelog topics for clarity; also use the source
topics num.partitions to set the num.partitions for repartition topics, and clarify to NOT
have cycles since otherwise the while loop will fail.
    
    3. `ensure-copartition` at the end to modify the number of partitions for repartition
topics if necessary to be equal to other co-partition topics.
    
    4. refactor `ClientState` as well and update the logic of `TaskAssignor` for clarity as
well.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/guozhangwang/kafka K4117-stream-partitionassignro-cleanup

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2012.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2012
    
----
commit fb0ad26e27cf3310d1ff9ad1c08f2d0e4e489496
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-07T23:03:52Z

    group client metadata

commit 2a81a8eedb9c7b16e1446b580c9c006afa78ffac
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T03:15:13Z

    cleanup co-partitioning process

commit f44d07cffc065f7abaeefcb5344484d44651dfb0
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T03:52:11Z

    re-order assignment logic

commit ec369149b036bfeaff4f743ef7f7bfe93819e641
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T03:56:28Z

    create changelog with compaction

commit 41e8c758390614dd8c66e29bbd92bb8d29bef26b
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T22:04:24Z

    fix unit test

commit 6296da70a17f1c8129a35189890ebb117a130b8f
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-11T15:38:04Z

    refactor client state

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message