kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yan Wang <yw...@xmatters.com>
Subject RE: Pause and max.poll.records
Date Mon, 26 Jun 2017 06:37:41 GMT
Sorry, seems it works as you said. System.out.print("+")  in my test is buffered. It does not
show the "+" immediately till I print a new line. Weird.

Thanks a lot!

Yan

-----Original Message-----
From: Yan Wang 
Sent: Sunday, June 25, 2017 11:18 PM
To: dev@kafka.apache.org
Subject: RE: Pause and max.poll.records

As I tested, seems if I pause p1 but do not call pause/resume again on p1, I cannot pull any
message from p2. But if I pause p1, I find I get one message from p2 each time I call pause
on p1 ( I mean I keep calling consumer.pause(p1) before consumer.poll).

Is it a bug or right behavior?

Thanks a lot!

Yan

-----Original Message-----
From: Jason Gustafson [mailto:jason@confluent.io]
Sent: Friday, June 23, 2017 3:08 PM
To: dev@kafka.apache.org
Subject: Re: Pause and max.poll.records

Fetches for each partition are sent once all the pending data for that partition has been
consumed. The only difference that pause makes is that we do not bother fetching paused partitions.
So in your example, p2 would be fetched even if p1 is paused and has fetched records >
max.poll.records pending.

-Jason

On Fri, Jun 23, 2017 at 1:43 PM, Yan Wang <ywang@xmatters.com> wrote:

> How the pause influences max.poll.records? If I have two partitions, 
> say
> p1 and p2. p1 has number local fetched records > max.poll.records. p2 
> has no data in local but in the server (not fetched yet). Now, if I pause P1.
> Does the next poll send fetch request to the server so that I can get 
> data for p2 from the server or we get nothing from p2 till we resume
> p1 and drain p1 till total local fetched records < max.poll.records?
>
>
> Thanks a lot!
>
Mime
View raw message