Return-Path: X-Original-To: apmail-storm-user-archive@minotaur.apache.org Delivered-To: apmail-storm-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1BEA610D59 for ; Thu, 3 Apr 2014 12:41:19 +0000 (UTC) Received: (qmail 49401 invoked by uid 500); 3 Apr 2014 12:41:18 -0000 Delivered-To: apmail-storm-user-archive@storm.apache.org Received: (qmail 48037 invoked by uid 500); 3 Apr 2014 12:41:15 -0000 Mailing-List: contact user-help@storm.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.incubator.apache.org Delivered-To: mailing list user@storm.incubator.apache.org Received: (qmail 47502 invoked by uid 99); 3 Apr 2014 12:41:12 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Apr 2014 12:41:12 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of kashyap.m@gmail.com designates 209.85.160.171 as permitted sender) Received: from [209.85.160.171] (HELO mail-yk0-f171.google.com) (209.85.160.171) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Apr 2014 12:41:07 +0000 Received: by mail-yk0-f171.google.com with SMTP id q9so1477270ykb.30 for ; Thu, 03 Apr 2014 05:40:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Gh+166JurECI81VkZZ3GtFWeusSvIndwFpMesE/ll5c=; b=iKvD6J2R3rcJUDGf+nZ6k9JaSIOWgndBwipzVxbU/j2ue+s+m1BQe1NTIony9W2nRb G+aQkBWcwFW8tPSjIOvoeooPUQtjqJCFnNYMitX+ULyUOVvbdM59u7V0sbs0kcDA+iLT LmeVpGJkvxKQfjO8JHsKFD/hdIxR2XX3bLqLw8MIZzbSzPPV/nHv289ZbP40YV5D6IZW y2L8T1ato9I6ltHl3D034Oaj/cN0ah9ABZ2LO5/XoZKZnXW2S/Phos24mCo9ig2izeBF m/t5WHfGdJaXjrv2sqIw5591LuTgK1vj7fxPT6WrRqwJk232RW5SiBnaqheJVOJuCRvR eMQw== MIME-Version: 1.0 X-Received: by 10.236.150.68 with SMTP id y44mr1414272yhj.113.1396528846379; Thu, 03 Apr 2014 05:40:46 -0700 (PDT) Received: by 10.170.176.3 with HTTP; Thu, 3 Apr 2014 05:40:46 -0700 (PDT) In-Reply-To: References: Date: Thu, 3 Apr 2014 07:40:46 -0500 Message-ID: Subject: Re: Kafka Rebalance exceptions in Storm From: Kashyap Mhaisekar To: "user@storm.incubator.apache.org" Content-Type: multipart/alternative; boundary=20cf303a3183cea05804f622b55b X-Virus-Checked: Checked by ClamAV on apache.org --20cf303a3183cea05804f622b55b Content-Type: text/plain; charset=ISO-8859-1 Thanks. How did you manage the offset persistence in this case? On Thursday, April 3, 2014, padma priya chitturi wrote: > The same exception i encountered when tried to integrate storm spout with > kafka high level consumer. This was resolved when i used SimpleConsumer. > > > On Thu, Apr 3, 2014 at 8:26 AM, Kashyap Mhaisekar > > wrote: > >> Hi, >> I consistently get the following exception when I submit a storm topology >> using StormSubmitter. I am using 0.8.2 Storm. >> The spout I use is a Kafka High Level consumer thats custom written. >> >> When I submit the same topology using LocalCluster, it works fine. Am I >> missing anything >> >> *kafka.common.ConsumerRebalanceFailedException: >> Sample_Kafka_client_abcd_com-1396492916541-2f28228c can't rebalance after 4 >> retries* >> * at >> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) >> ~[stormjar.jar:na]* >> * at >> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722) >> ~[stormjar.jar:na]* >> * at >> kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:212) >> ~[stormjar.jar:na]* >> * at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:80) >> ~[stormjar.jar:na]* >> * at >> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:92) >> ~[stormjar.jar:na]* >> >> >> *2014-04-02 21:41:58 b.s.util [INFO] Halting process: ("Worker died") * >> >> Thanks in advance >> Regards, >> Kashyap >> > > --20cf303a3183cea05804f622b55b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Thanks. How did you manage the offset persistence in this case?

On T= hursday, April 3, 2014, padma priya chitturi <padmapriya30@gmail.com> wrote:
The same exception i encountered when tried to integrate s= torm spout with kafka high level consumer. This was resolved when i used Si= mpleConsumer.


On Thu, Apr 3, 2014 at 8:26 AM, Kashyap Mhaisekar <kashyap.m@gmail.com> wrote:
Hi,
I consistently get the following except= ion when I submit a storm topology using StormSubmitter. I am using 0.8.2 S= torm.
The spout I use is a Kafka High Level consumer thats custom= written.

When I submit the same topology using LocalCluster, it = works fine. Am I missing anything

kafka.common.= ConsumerRebalanceFailedException: Sample_Kafka_client_abcd_com-139649291654= 1-2f28228c can't rebalance after 4 retries
=A0 =A0 =A0 =A0 at kafka.consumer.ZookeeperConsumerConnector$ZKReba= lancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432) ~[stor= mjar.jar:na]
=A0 =A0 =A0 =A0 at kafka.consumer.ZookeeperCo= nsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeCons= umer(ZookeeperConsumerConnector.scala:722) ~[stormjar.jar:na]
=A0 =A0 =A0 =A0 at kafka.consumer.ZookeeperConsumerConnector.consum= e(ZookeeperConsumerConnector.scala:212) ~[stormjar.jar:na]
=A0 =A0 =A0 =A0 at kafka.javaapi.consumer.ZookeeperConsumerConnector.creat= eMessageStreams(ZookeeperConsumerConnector.scala:80) ~[stormjar.jar:na]=
=A0 =A0 =A0 =A0 at kafka.javaapi.consumer.ZookeeperConsumerConnecto= r.createMessageStreams(ZookeeperConsumerConnector.scala:92) ~[stormjar.jar:= na]

2014-04-02 21:41:58 b.s.util [IN= FO] Halting process: ("Worker died")

Thanks in advance
Regards,
Kashyap

--20cf303a3183cea05804f622b55b--