Mailing-List: contact users-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@kafka.apache.org
Received-SPF: pass (athena.apache.org: domain of libo.yu@citi.com designates
 67.231.145.106 as permitted sender)
From: "Yu, Libo " <libo.yu@citi.com>
To: "'users@kafka.apache.org'" <users@kafka.apache.org>
Subject: RE: how brokers sync with each other
Thread-Topic: how brokers sync with each other
Thread-Index: AQHOdGSG2gjoQKK/SPaPxe0B2iiC9plM96Yw
Date: Sat, 29 Jun 2013 17:47:08 +0000
Message-ID: <FF142F6B499AE34CAED4D263F6CA32901D32AA77@EXTXMB19.nam.nsroot.net>
References: <FF142F6B499AE34CAED4D263F6CA32901D32AA14@EXTXMB19.nam.nsroot.net>
 <CAAOfhrAhqqbuaMO2sv99EHOzP+cmcG=9a4-A0qg26JV6jxjrDg@mail.gmail.com>
In-Reply-To: 
 <CAAOfhrAhqqbuaMO2sv99EHOzP+cmcG=9a4-A0qg26JV6jxjrDg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Thanks for your answer, Joel. What parameters can I tune in this case to in=
crease the lag?
By default, under what kind of scenario will a replica it fall out of the "=
in-sync-replica"
Set?

Regards,

Libo


-----Original Message-----
From: Joel Koshy [mailto:jjkoshy.w@gmail.com]=20
Sent: Friday, June 28, 2013 9:04 PM
To: users@kafka.apache.org
Subject: Re: how brokers sync with each other

I'm assuming this is somewhat related to your previous question on cross-DC=
 replication. This is not an ideal set up as mentioned there.
If the replica lags then it will fall out of the "in-sync-replica"
set. You could tune parameters that effectively allow a high (but bounded o=
ver time) lag between the replica and the leader. There is no buffer per se=
 - the replicas just keep fetching from the leader (which would have append=
ed the messages to its log).

If your producer has acks set to -1 (which means wait until all replicas in=
 ISR have received the message) it would slow down your producer considerab=
ly. (You could use acks =3D 1 in that case though but with weaker guarantee=
s - i.e., your message could be lost in failure scenarios).

Joel


On Fri, Jun 28, 2013 at 5:41 PM, Yu, Libo <libo.yu@citi.com> wrote:
> Hi,
>
> Assume A and B are two brokers in a kafka server. And there is long=20
> network Latency between A and B. For a partition with two=20
> replications, one replication Is assigned to A and the other is=20
> assigned to B. Number of acknowledge is set to one. Assume the partition =
is handled by broker A.
>
> After a message is accepted and saved by A, ack will be sent and the=20
> message will be sent to broker B at the same time. But due to long=20
> network latency between A and B, it is possible that B lags behind A.=20
> In this case, is there a buffer on broker A that holds all messages to=20
> be written to B? Is there any limit on the lag between A and B?
>
> Compare with the case when there is no network latency between A and=20
> B, any difference in publishing speed?
>
> Thanks
>
> Libo
>