kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danny Yeshurun <da...@myheritage.com>
Subject Re: Response time degradation
Date Tue, 29 Jan 2013 10:40:30 GMT
Will do.

Regarding the multi-fetch request - say I give a list of topic-partition,
is it guaranteed that all pairs will be "scanned", i.e. if there is a
message in a partition I will always get it in one call?

Thanks,
Danny

On Mon, Jan 28, 2013 at 7:54 PM, Neha Narkhede <neha.narkhede@gmail.com>wrote:

> Danny,
>
> Please can you file a bug to explain the issue ?
>
> Thanks,
> Neha
>
>
> On Mon, Jan 28, 2013 at 8:57 AM, Danny Yeshurun <danny@myheritage.com
> >wrote:
>
> > No data on the second time. The partition is empty but starting from the
> > second request the response is slow.
> >
> > On Mon, Jan 28, 2013 at 6:31 PM, Jun Rao <junrao@gmail.com> wrote:
> >
> > > Not sure what's the causing the slow down. Is the second fetch geting
> > more
> > > data?
> > >
> > > Also, if all those partitions are on the same broker, you could have
> > > fetched all partitions' data in a single fetch request.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Jan 28, 2013 at 6:02 AM, Danny Yeshurun <danny@myheritage.com
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > I am using Kafka 0.7.2
> > > > I have a consumer that tries to fetch messages from several
> partitions
> > on
> > > > the same broker (and topic).
> > > > The consumers issues a fetch request per partition, if there are
> > messages
> > > > to read, it fetches the first one and goes to the next partition to
> > fetch
> > > > another message and so on (in a loop).
> > > > When there are not messages in a partition, the consumer will issue
> > > another
> > > > fetch request on the next cycle etc.
> > > > I noticed that starting with the 2nd fetch request, response time
> > becomes
> > > > unacceptable, i.e. reading back the response from the broker takes
> > ~0.04
> > > > second (while it took ~0.0006 sec on the first time).
> > > > This behavior is consistent over different runs.
> > > >
> > > > Any idea?
> > > >
> > > > Thanks,
> > > > Danny
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message