kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5212) Consumer ListOffsets request can starve group heartbeats
Date Wed, 11 Oct 2017 04:05:02 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16199778#comment-16199778
] 

Richard Yu commented on KAFKA-5212:
-----------------------------------

During some study after the failure of my most recent commit, there is this approach which
one would note:

1. Due to hierarchy design, heartbeat is only accessible by ConsumerCoordinator.
2. In order to pass a variable to Fetcher, ConsumerCoordinator must pass the necessary information
through ConsumerNetworkClient.

One of the easiest ways to is to define a variable with the time of the next heartbeat (e.g.
long nextHeartbeat = now + remainingMs). The variable is defined as a private field of ConsumerNetworkClient
but is defined by ConsumerCoordinator's methods. This way, Fetcher could have access to the
next heartbeat time.

However, this is when we encounter a problem: If fetcher's poll() is concluded (it could not
yield, thus we must wait), and if the time.milliseconds() exceeds nextHeartbeat, we must resend
a heartbeat. However, the poll() method is currently only accessible in ConsumerCoordinator.
Is there a way to get around this?

> Consumer ListOffsets request can starve group heartbeats
> --------------------------------------------------------
>
>                 Key: KAFKA-5212
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5212
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>            Reporter: Jason Gustafson
>            Assignee: Richard Yu
>             Fix For: 1.1.0, 1.0.1
>
>
> The consumer is not able to send heartbeats while it is awaiting a ListOffsets response.
Typically this is not a problem because ListOffsets requests are handled quickly, but in the
worst case if the request takes longer than the session timeout, the consumer will fall out
of the group.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message