Return-Path: X-Original-To: apmail-incubator-kafka-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-kafka-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 70EC5BC2F for ; Fri, 6 Jan 2012 16:32:14 +0000 (UTC) Received: (qmail 99837 invoked by uid 500); 6 Jan 2012 16:32:14 -0000 Delivered-To: apmail-incubator-kafka-dev-archive@incubator.apache.org Received: (qmail 99760 invoked by uid 500); 6 Jan 2012 16:32:13 -0000 Mailing-List: contact kafka-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: kafka-dev@incubator.apache.org Delivered-To: mailing list kafka-dev@incubator.apache.org Received: (qmail 99746 invoked by uid 99); 6 Jan 2012 16:32:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jan 2012 16:32:13 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of junrao@gmail.com designates 209.85.212.175 as permitted sender) Received: from [209.85.212.175] (HELO mail-wi0-f175.google.com) (209.85.212.175) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jan 2012 16:32:08 +0000 Received: by wibhq7 with SMTP id hq7so1333972wib.6 for ; Fri, 06 Jan 2012 08:31:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=GPqFsk3G6VdjajkAv4tz2IcH0tyf5VGbxJVFKZrT3UQ=; b=IPE2bgryLNX6yk59CaJKdcij8DMoaY0zYur0T3q+p/2onyU+GRbCAJ/bygP5O/G4OE 67zmJRymfhp/8z7BoHursa+zqOhBZraDWBGpbIVUTtZ1UdZGMjnp5VH3AuYMje7wbwoK a3CC57DfcSdCm2ae3ILhTk42GY/o6h16C6QRc= MIME-Version: 1.0 Received: by 10.180.19.130 with SMTP id f2mr8408258wie.12.1325867507320; Fri, 06 Jan 2012 08:31:47 -0800 (PST) Received: by 10.216.37.67 with HTTP; Fri, 6 Jan 2012 08:31:47 -0800 (PST) In-Reply-To: References: Date: Fri, 6 Jan 2012 08:31:47 -0800 Message-ID: Subject: Re: skype meeting on replication roadmap From: Jun Rao To: kafka-users@incubator.apache.org, kafka-dev@incubator.apache.org Content-Type: multipart/alternative; boundary=bcaec53f37edcb08d504b5de9573 --bcaec53f37edcb08d504b5de9573 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Joe, I am ok with moving the meeting to Monday. If nobody objects, how about we meet at 2pm PST Monday (Jan 9) instead? This actually will give people a bit more time to read the docs and wiki. Jun On Fri, Jan 6, 2012 at 6:38 AM, Joe Stein wrote: > so it looks like i need to take my son to a birthday party tonight (my ba= be > daughter is sick wife has to stay home with her) > > if there is any chance to move this to Monday? (sorry) if not I understan= d > either way let me know > > On Thu, Jan 5, 2012 at 2:11 PM, Jun Rao wrote: > > > Hi, everyone, > > > > So far, the following people have sent me their skype id: > > > > Joe, David, Jeffrey, and Andrew. > > > > So the meeting is on. I encourage everyone to look at the design doc in > > kafka-50 and the new wire format at > > > > > https://cwiki.apache.org/confluence/display/KAFKA/New+Wire+Format+Proposa= lbefore > > the meeting. > > > > Here is a breakdown of all the jiras and their dependencies related to > the > > replication work: > > > > 1. kafka-47: create/delete data structures in ZK, automatically create > > topic, and use them (making partitions logical, supporting only 1 > replica, > > no failure support). =EF=BB=BF=EF=BB=BF(L1) > > 1.1 kafka-237: create/delete ZK path for a topic in an admin tool (L0) > > 1.2 kafka-238: add a getTopicMetaData method in broker and expose it t= o > > producer > > 1.3 kafka-239: Wire existing producer and consumer to use the new ZK > data > > structure > > 2. kafka-202: decouple request handler and socket sever; enabling long > poll > > in the consumer (L1) > > 3. kafka-240: implement new producer and consumer request format (L1) > > 4. kafka-49: add ack to ProduceRequest (L2). depending on #3 > > 5. kafka-48: long poll in consumer (L2). depending on #2 > > 6. kafka-44: commit thread, replica fetcher thread (L3). depending on #= 1, > > #4, #5 > > 7. kafka-45: broker starting up, leader election (L3). depending on #1 > > 8. kafka-46: various ZK listeners (L3). depending on #1 > > 9. kafka-43: move master to preferred replica when possible (L4). > > optimization > > 10. kafka-42: rebalance partition with new brokers (L4). extra feature > > > > Thanks, > > > > Jun > > > > On Wed, Jan 4, 2012 at 10:11 AM, Jun Rao wrote: > > > > > Hi, > > > > > > We'd like have a meeting discussing the roadmap for replication (our > 0.8 > > > release) and how people can contribute. How about a Skype meeting at > 2pm > > > Jan 6 (Friday)? If you are interested, please send me your Skype id. > > > > > > Thanks, > > > > > > Jun > > > > > > > > > -- > > /* > Joe Stein > http://www.linkedin.com/in/charmalloc > Twitter: @allthingshadoop > */ > --bcaec53f37edcb08d504b5de9573--