Return-Path: <dev-return-64609-archive-asf-public=cust-asf.ponee.io@kafka.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 6F0E6200B9F
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Tue, 11 Oct 2016 17:48:56 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 6DD43160AE6; Tue, 11 Oct 2016 15:48:56 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id B6FA9160AC3
	for <archive-asf-public@cust-asf.ponee.io>; Tue, 11 Oct 2016 17:48:55 +0200 (CEST)
Received: (qmail 50564 invoked by uid 500); 11 Oct 2016 15:48:54 -0000
Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@kafka.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@kafka.apache.org>
List-Post: <mailto:dev@kafka.apache.org>
List-Id: <dev.kafka.apache.org>
Reply-To: dev@kafka.apache.org
Delivered-To: mailing list dev@kafka.apache.org
Received: (qmail 50553 invoked by uid 99); 11 Oct 2016 15:48:54 -0000
Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Oct 2016 15:48:54 +0000
Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33)
	id 19D26E0061; Tue, 11 Oct 2016 15:48:54 +0000 (UTC)
From: guozhangwang <git@git.apache.org>
To: dev@kafka.apache.org
Reply-To: dev@kafka.apache.org
Message-ID: <git-pr-2012-kafka@git.apache.org>
Subject: [GitHub] kafka pull request #2012: [WIP] KAFKA-4117: Stream partitionassignro cleanup
Content-Type: text/plain
Date: Tue, 11 Oct 2016 15:48:54 +0000 (UTC)
archived-at: Tue, 11 Oct 2016 15:48:56 -0000

GitHub user guozhangwang opened a pull request:

    https://github.com/apache/kafka/pull/2012

    [WIP] KAFKA-4117: Stream partitionassignro cleanup

    1. Create a new ClientMetadata` to collapse `Set<String> consumerMemberIds`, `ClientState<TaskId> state`, and `HostInfo hostInfo`.
    
    2. Stop reusing `stateChangelogTopicToTaskIds` and `internalSourceTopicToTaskIds` to access the (sub-)topology's internal repartition and changelog topics for clarity; also use the source topics num.partitions to set the num.partitions for repartition topics, and clarify to NOT have cycles since otherwise the while loop will fail.
    
    3. `ensure-copartition` at the end to modify the number of partitions for repartition topics if necessary to be equal to other co-partition topics.
    
    4. refactor `ClientState` as well and update the logic of `TaskAssignor` for clarity as well.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/guozhangwang/kafka K4117-stream-partitionassignro-cleanup

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2012.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2012
    
----
commit fb0ad26e27cf3310d1ff9ad1c08f2d0e4e489496
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-07T23:03:52Z

    group client metadata

commit 2a81a8eedb9c7b16e1446b580c9c006afa78ffac
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T03:15:13Z

    cleanup co-partitioning process

commit f44d07cffc065f7abaeefcb5344484d44651dfb0
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T03:52:11Z

    re-order assignment logic

commit ec369149b036bfeaff4f743ef7f7bfe93819e641
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T03:56:28Z

    create changelog with compaction

commit 41e8c758390614dd8c66e29bbd92bb8d29bef26b
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-08T22:04:24Z

    fix unit test

commit 6296da70a17f1c8129a35189890ebb117a130b8f
Author: Guozhang Wang <wangguoz@gmail.com>
Date:   2016-10-11T15:38:04Z

    refactor client state

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---