Return-Path: X-Original-To: apmail-incubator-kafka-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-kafka-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 563D99CBF for ; Fri, 6 Jan 2012 00:59:03 +0000 (UTC) Received: (qmail 88909 invoked by uid 500); 6 Jan 2012 00:59:03 -0000 Delivered-To: apmail-incubator-kafka-dev-archive@incubator.apache.org Received: (qmail 88883 invoked by uid 500); 6 Jan 2012 00:59:02 -0000 Mailing-List: contact kafka-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: kafka-dev@incubator.apache.org Delivered-To: mailing list kafka-dev@incubator.apache.org Received: (qmail 88875 invoked by uid 99); 6 Jan 2012 00:59:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jan 2012 00:59:02 +0000 X-ASF-Spam-Status: No, hits=0.3 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_SOFTFAIL X-Spam-Check-By: apache.org Received-SPF: softfail (athena.apache.org: transitioning domain of list@toolazydogs.com does not designate 209.85.210.175 as permitted sender) Received: from [209.85.210.175] (HELO mail-iy0-f175.google.com) (209.85.210.175) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jan 2012 00:58:54 +0000 Received: by iakh37 with SMTP id h37so1767983iak.6 for ; Thu, 05 Jan 2012 16:58:34 -0800 (PST) Received: by 10.50.40.133 with SMTP id x5mr5477197igk.29.1325811514476; Thu, 05 Jan 2012 16:58:34 -0800 (PST) Received: from [172.18.37.30] (dagmar.corp.linkedin.com. [69.28.149.29]) by mx.google.com with ESMTPS id py4sm126489804igc.2.2012.01.05.16.58.32 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 05 Jan 2012 16:58:33 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Apple Message framework v1251.1) Subject: Re: skype meeting on replication roadmap From: "Alan D. Cabrera" In-Reply-To: Date: Thu, 5 Jan 2012 16:58:30 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: kafka-dev@incubator.apache.org X-Mailer: Apple Mail (2.1251.1) I need to point out my personal understanding of the parameters of a = conference call meeting. Conference calls can be useful to facilitate = design discussions though their use is not encouraged at ASF. It's = important that participants understand and abide by the following = guidelines: - while consensus can be built at the call no final decision can be made - meeting minutes are posted to the mailing lists - an adequate effort is made to give the larger community the ability to = participate in a meaningful way after the call Regards, Alan On Jan 5, 2012, at 11:11 AM, Jun Rao wrote: > Hi, everyone, >=20 > So far, the following people have sent me their skype id: >=20 > Joe, David, Jeffrey, and Andrew. >=20 > So the meeting is on. I encourage everyone to look at the design doc = in > kafka-50 and the new wire format at > = https://cwiki.apache.org/confluence/display/KAFKA/New+Wire+Format+Proposal= before > the meeting. >=20 > Here is a breakdown of all the jiras and their dependencies related to = the > replication work: >=20 > 1. kafka-47: create/delete data structures in ZK, automatically create > topic, and use them (making partitions logical, supporting only 1 = replica, > no failure support). =EF=BB=BF=EF=BB=BF(L1) > 1.1 kafka-237: create/delete ZK path for a topic in an admin tool = (L0) > 1.2 kafka-238: add a getTopicMetaData method in broker and expose it = to > producer > 1.3 kafka-239: Wire existing producer and consumer to use the new ZK = data > structure > 2. kafka-202: decouple request handler and socket sever; enabling long = poll > in the consumer (L1) > 3. kafka-240: implement new producer and consumer request format (L1) > 4. kafka-49: add ack to ProduceRequest (L2). depending on #3 > 5. kafka-48: long poll in consumer (L2). depending on #2 > 6. kafka-44: commit thread, replica fetcher thread (L3). depending on = #1, > #4, #5 > 7. kafka-45: broker starting up, leader election (L3). depending on #1 > 8. kafka-46: various ZK listeners (L3). depending on #1 > 9. kafka-43: move master to preferred replica when possible (L4). > optimization > 10. kafka-42: rebalance partition with new brokers (L4). extra feature >=20 > Thanks, >=20 > Jun >=20 > On Wed, Jan 4, 2012 at 10:11 AM, Jun Rao wrote: >=20 >> Hi, >>=20 >> We'd like have a meeting discussing the roadmap for replication (our = 0.8 >> release) and how people can contribute. How about a Skype meeting at = 2pm >> Jan 6 (Friday)? If you are interested, please send me your Skype id. >>=20 >> Thanks, >>=20 >> Jun >>=20