Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Mon, 12 Oct 2015 08:21:05 +0000 (UTC)
From: "Sreenivasulu Nallapati (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12895398.1442917994000.11795.1444638065481@Atlassian.JIRA>
In-Reply-To: <JIRA.12895398.1442917994000@Atlassian.JIRA>
References: <JIRA.12895398.1442917994000@Atlassian.JIRA>
 <JIRA.12895398.1442917994951@arcas>
Subject: [jira] [Commented] (KAFKA-2565) Offset Commit is not working if
 multiple consumers try to commit the offset
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/KAFKA-2565?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1495=
2795#comment-14952795 ]=20

Sreenivasulu Nallapati commented on KAFKA-2565:
-----------------------------------------------

This happens basically in this scenario.
Our batch works as below
1. Even though we are running our consumer in batch mode, we are using High=
 Level Consumer for scalability.
2. Opens kafka consumer connector
3. Start of the consumer  it will identify the latest offset(consumer stop =
offset -cso) for each partition
4. It will start reading messages till the cso for this batch.
5. Write the messages to a temp files on FTP server. Once we process all th=
e messages, move the FTP server temp files to actual location on FTP server
6. If all data transfer is success, commit the offset to zookeeper.
7. If we run Single consumer to process all the partitions, there is no iss=
ue :)
8. If we start multiple consumers for a single topic ( say three consumers,=
 three partitions. one consumer for one partition) the problem starts.
What we observed here is: Out of three consumers if one consumer(c1-partiti=
on1) finishes its processing ahead of other two. The zookeeper sees a re ba=
lancing and start re balancing partition1 with one of other two running con=
sumers(while zookeeper doing this task other consumers consumed all the mes=
sages and in the process of moving the temp files on FTP server). We are no=
t closing the consumer connector till end of the batch. The re balancing is=
 happening after we stopped consuming the message.

Is there something we are missing here or doing wrong


> Offset Commit is not working if multiple consumers try to commit the offs=
et
> -------------------------------------------------------------------------=
--
>
>                 Key: KAFKA-2565
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2565
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions: 0.8.1, 0.8.2.1, 0.8.2.2
>            Reporter: Sreenivasulu Nallapati
>            Assignee: Neha Narkhede
>
> We are seeing some strange behaviour with commitOffsets() method of kafka=
.javaapi.consumer.ConsumerConnector. We committing the offsets to zookeeper=
 at the end of the consumer batch. We are running multiple consumers for th=
e same topic.
> Test details:=20
> 1.=09Created a topic with three partitions
> 2.=09Started three consumers (cronjob) at the same time. The aim is that =
each consumer to process one partition.
> 3.=09Each consumer at the end of the batch, it will call the commitOffset=
s() method on kafka.javaapi.consumer.ConsumerConnector
> 4.=09The offsets are getting properly updated in zookeeper if we run the =
consumers for small set (say 1000 messages) of messages.
> 5.=09But for larger number of messages, commit offset is not working as e=
xpected=E2=80=A6sometimes only two offsets are properly committing and othe=
r one remains as it was.
> 6.=09Please see the below example
> Partition: 0 Latest Offset: 1057585
> Partition: 1 Latest Offset: 1057715
> Partition: 2 Latest Offset: 1057590
> Earliest Offset after all consumers completed: {0=3D1057585, 1=3D724375, =
2=3D1057590}
> Highlighted in red supposed to be committed as 1057715 but it did not.
> Please check if it is bug with multiple consumers. When multiple consumer=
s are trying to update the same path in Zookeper, is there any synchronizat=
ion issue?
> Kafka Cluster details
> 1 zookeeper
> 3 brokers


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)