kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Kreps (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-2400) Expose heartbeat frequency in new consumer configuration
Date Mon, 03 Aug 2015 22:24:05 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652663#comment-14652663
] 

Jay Kreps commented on KAFKA-2400:
----------------------------------

Also we should set good defaults: 
- session timeout should probably default to something pretty high, this will mean longer
time to detect true failures but no false positives or churning, those who want faster detection
can tune down appropriately (most won't care)
- reasonable heartbeat frequency (300 ms?).

> Expose heartbeat frequency in new consumer configuration
> --------------------------------------------------------
>
>                 Key: KAFKA-2400
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2400
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Jason Gustafson
>            Assignee: Jason Gustafson
>            Priority: Minor
>
> The consumer coordinator communicates the need to rebalance through responses to heartbeat
requests sent from each member of the consumer group. The heartbeat frequency therefore controls
how long normal rebalances will take. Currently, the frequency is hard-coded to 3 heartbeats
per the configured session timeout, but it would be nice to expose this setting so that the
user can control the impact from rebalancing.
> Since the consumer is currently single-threaded and heartbeats are sent in poll(), we
cannot guarantee that the heartbeats will actually be sent at the configured frequency. In
practice, the user may have to adjust their fetch size to ensure that poll() is called often
enough to get the desired heartbeat frequency. For most users, the consumption rate is probably
fast enough for this not to matter, but we should make the documentation clear on this point.
In any case, we expect that most users will accept the default value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message