kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu, Libo " <libo...@citi.com>
Subject RE: how brokers sync with each other
Date Sat, 29 Jun 2013 17:47:08 GMT
Thanks for your answer, Joel. What parameters can I tune in this case to increase the lag?
By default, under what kind of scenario will a replica it fall out of the "in-sync-replica"



-----Original Message-----
From: Joel Koshy [mailto:jjkoshy.w@gmail.com] 
Sent: Friday, June 28, 2013 9:04 PM
To: users@kafka.apache.org
Subject: Re: how brokers sync with each other

I'm assuming this is somewhat related to your previous question on cross-DC replication. This
is not an ideal set up as mentioned there.
If the replica lags then it will fall out of the "in-sync-replica"
set. You could tune parameters that effectively allow a high (but bounded over time) lag between
the replica and the leader. There is no buffer per se - the replicas just keep fetching from
the leader (which would have appended the messages to its log).

If your producer has acks set to -1 (which means wait until all replicas in ISR have received
the message) it would slow down your producer considerably. (You could use acks = 1 in that
case though but with weaker guarantees - i.e., your message could be lost in failure scenarios).


On Fri, Jun 28, 2013 at 5:41 PM, Yu, Libo <libo.yu@citi.com> wrote:
> Hi,
> Assume A and B are two brokers in a kafka server. And there is long 
> network Latency between A and B. For a partition with two 
> replications, one replication Is assigned to A and the other is 
> assigned to B. Number of acknowledge is set to one. Assume the partition is handled by
broker A.
> After a message is accepted and saved by A, ack will be sent and the 
> message will be sent to broker B at the same time. But due to long 
> network latency between A and B, it is possible that B lags behind A. 
> In this case, is there a buffer on broker A that holds all messages to 
> be written to B? Is there any limit on the lag between A and B?
> Compare with the case when there is no network latency between A and 
> B, any difference in publishing speed?
> Thanks
> Libo

View raw message