Return-Path: X-Original-To: apmail-kafka-users-archive@www.apache.org Delivered-To: apmail-kafka-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0BA46E3B0 for ; Fri, 8 Feb 2013 09:41:33 +0000 (UTC) Received: (qmail 62837 invoked by uid 500); 8 Feb 2013 09:41:32 -0000 Delivered-To: apmail-kafka-users-archive@kafka.apache.org Received: (qmail 62646 invoked by uid 500); 8 Feb 2013 09:41:32 -0000 Mailing-List: contact users-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@kafka.apache.org Delivered-To: mailing list users@kafka.apache.org Received: (qmail 62605 invoked by uid 99); 8 Feb 2013 09:41:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Feb 2013 09:41:31 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of michal.haris@imagini.net designates 209.85.215.41 as permitted sender) Received: from [209.85.215.41] (HELO mail-la0-f41.google.com) (209.85.215.41) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Feb 2013 09:41:25 +0000 Received: by mail-la0-f41.google.com with SMTP id fo12so3591853lab.0 for ; Fri, 08 Feb 2013 01:41:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:x-originating-ip:in-reply-to:references :date:message-id:subject:from:to:content-type:x-gm-message-state; bh=Q+KhZ+DVzYG61asUNwT7+Y5ZzwZqaWJLdQoPLls8/8o=; b=oBWi2yxawDyua0QI7fZxE4NxQxWiG6OC20/JAa/7Pc824b/74c12QmzT6cxl5Gf4eJ SPN2aibabu1iDKxiJpQiRVxEK/IR2bzG8aUUlj8Do0y6uAULQva7rjCPVFzEwdE85poq vmsXic0s28OwOzICsY7gtFVlrCj3HVuhK635HHeo+VnwfYr34owMTuG2/pET+uP9L82Z 17MeHPJcO73/EX746KhcpItlI2QTDNwnbSNnXsnNxQtj6Fs0xCJ4uV9J6n1cQ/lRS2h9 7vFun2kz2Z0XbWwcvm4qG83ZdRM0jkCzwT/Fm2Oq7CrreJAo3tPWKbPFpEPOhuULoeFS 2JQQ== MIME-Version: 1.0 X-Received: by 10.112.16.102 with SMTP id f6mr2041619lbd.3.1360316463129; Fri, 08 Feb 2013 01:41:03 -0800 (PST) Received: by 10.112.54.141 with HTTP; Fri, 8 Feb 2013 01:41:02 -0800 (PST) X-Originating-IP: [82.71.191.141] Received: by 10.112.54.141 with HTTP; Fri, 8 Feb 2013 01:41:02 -0800 (PST) In-Reply-To: References: Date: Fri, 8 Feb 2013 09:41:02 +0000 Message-ID: Subject: Re: kafka replication blog From: Michal Haris To: users@kafka.apache.org Content-Type: multipart/alternative; boundary=f46d0401fd0f91238c04d5335bc3 X-Gm-Message-State: ALoCoQmwgQQo82KaA5WqaMMhCqWqfpZgxZWVSvf6LSRPQPw3X2NJniIdT2bd+fjqR++EnE3uekAq X-Virus-Checked: Checked by ClamAV on apache.org --f46d0401fd0f91238c04d5335bc3 Content-Type: text/plain; charset=ISO-8859-1 So if the produces are partitioning by key we have to have replication if we dont want messages to get lost when partition goes down l right ? Thanks On Feb 8, 2013 5:12 AM, "Jun Rao" wrote: > We have fixed this issue in 0.8. Withreplication factor 1, if the producer > doesn't care about partitioning by key, messages will be sent to partitions > that are currently available. > > Thanks, > > Jun > > On Thu, Feb 7, 2013 at 3:11 PM, Michal Haris >wrote: > > > Same here, summary was need as we have a fairly large ecosystem of > multiple > > 0.7.2 clusters and I am planning to test upgrade to 0.8. > > However, one thing creeping at the back of my mind regarding 0.8 is > > something i have spotted in one thread few weeks ago namely that the > > rebalance behaviour of producers is not as robust as in 0.7.x without > > replication and i remeber there was no designed solution at the time - > any > > news here ? Basically our usecase doesn't require replication but logical > > offsets and some other things introduced would solve some problems. > > On Feb 7, 2013 7:11 PM, "Vaibhav Puranik" wrote: > > > > > Same here. Thanks a lot Jun. > > > > > > Regards, > > > Vaibhav > > > > > > On Thu, Feb 7, 2013 at 10:38 AM, Felix GV wrote: > > > > > > > Thanks Jun! > > > > > > > > I hadn't been following the discussions regarding 0.8 and replication > > > for a > > > > little while and this was a great post to refresh my memory and get > up > > to > > > > speed on the current replication architecture's design. > > > > > > > > -- > > > > Felix > > > > > > > > > > > > On Tue, Feb 5, 2013 at 2:21 PM, Jun Rao wrote: > > > > > > > > > I just posted the following blog on Kafka replication. This may > > answer > > > > some > > > > > of the questions that a few people have asked in the mailing list > > > before. > > > > > > > > > > > > > > > > > > > > > > > > > http://engineering.linkedin.com/kafka/intra-cluster-replication-apache-kafka > > > > > > > > > > Thanks, > > > > > > > > > > Jun > > > > > > > > > > > > > > > --f46d0401fd0f91238c04d5335bc3--