kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 王国栋 <wangg...@gmail.com>
Subject Re: ASyncProducer loss more messages than SyncProducer when broker crashes.
Date Tue, 05 Feb 2013 12:54:14 GMT
Thanks Jun.


On Tue, Feb 5, 2013 at 5:03 PM, Jun Rao <junrao@gmail.com> wrote:

> It could be. This is because it takes ZK up to 6 secs to detect the failure
> due to a hard kill (since ZK client is not closed properly). You can shrink
> ZK session timeout to reduce the delay.
>
> Thanks,
>
> Jun
>
> On Mon, Feb 4, 2013 at 10:30 PM, 王国栋 <wanggd04@gmail.com> wrote:
>
> > We kill the broker on purpose, since we are doing some fail over tests.
> If
> > the broker is hard killed, will the async-producer lose message for 6
> > seconds?
> >
> > Thanks.
> >
> > Guodong
> >
> >
> > On Tue, Feb 5, 2013 at 2:19 PM, Jun Rao <junrao@gmail.com> wrote:
> >
> > > Is the broker shut down cleanly? If so, the producer should get a ZK
> > > notification immediately. If the broker is hard killed, the ZK
> > notification
> > > won't arrive at the producer until zk session timeout (default is 6
> > secs).
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Feb 4, 2013 at 7:11 PM, 王国栋 <wanggd04@gmail.com> wrote:
> > >
> > > > Hi Jun
> > > >
> > > > We are using zookeeper. And the async-producer loses  messages in 'X'
> > > > seconds. Here, X is between 4 and 6. It seems that X is comparable
> with
> > > > zookeeper session timeout.
> > > >
> > > > I am wondering the behavior of async-producer when socket is broken,
> > but
> > > > the broker is still visible for zookeeper client.
> > > >
> > > >
> > > > On Tue, Feb 5, 2013 at 12:39 AM, Jun Rao <junrao@gmail.com> wrote:
> > > >
> > > > > Currently, there is no callback to report socket failure in the
> async
> > > > > producer. However, async producer should be able to detect socket
> > > failure
> > > > > and reconnect to another broker. Are you using a vip or Zookeeper
> in
> > > the
> > > > > producer for load balancing?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Sun, Feb 3, 2013 at 10:00 PM, 王国栋 <wanggd04@gmail.com>
wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > we are doing some fail over test on Kafka0.7.2. We kill one
of
> the
> > > > > brokers
> > > > > > on purpose. But we find that the async-producer loss more
> messages.
> > > We
> > > > do
> > > > > > catch exceptions for send api. But it seems that the
> async-producer
> > > > > doesn't
> > > > > > throw any exceptions when broker is done.
> > > > > >
> > > > > > We change the producer to sync-producer, and got the exceptions
> as
> > > soon
> > > > > as
> > > > > > the broker is down.
> > > > > >
> > > > > > So I am wondering if I can get the socket status connected to
the
> > > > broker.
> > > > > > Does some callbacks can do this?
> > > > > >
> > > > > > Thanks.
> > > > > > --
> > > > > > Guodong Wang
> > > > > > 王国栋
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Guodong Wang
> > > > 王国栋
> > > >
> > >
> >
> >
> >
> > --
> > Guodong Wang
> > 王国栋
> >
>



-- 
Guodong Wang
王国栋

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message