kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4558) throttling_test fails if the producer starts too fast.
Date Mon, 09 Jan 2017 03:56:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15810593#comment-15810593

Ewen Cheslack-Postava commented on KAFKA-4558:

[~apurva] I couldn't remember when lag metrics are actually updated, but I guess if they are
0 even if the topic doesn't exist then it doesn't help. You're right that at least we'll be
a lot closer to the correct condition by checking that we at least have an assignment.

Using JMX seems fine to me, I didn't look carefully through the JmxMixin implementation but
thought I remembered it being pretty messy (I think partially since it has to run an extra
process that does the data collection). If it's easy to get from that, that's great. I was
also selfishly considering other future uses of metrics where we'd probably like there to
be easier (and cheaper) ways to capture this info :)

Re: HttpMetricsReporter -- nothing in tests requires a KIP since it'd be unsupported, although
I have no doubt once it existed some folks would want it to be supported.

> throttling_test fails if the producer starts too fast.
> ------------------------------------------------------
>                 Key: KAFKA-4558
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4558
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Apurva Mehta
>            Assignee: Apurva Mehta
> As described in https://issues.apache.org/jira/browse/KAFKA-4526, the throttling test
will fail if the producer in the produce-consume-validate loop starts up before the consumer
is fully initialized.
> We need to block the start of the producer until the consumer is ready to go. 
> The current plan is to poll the consumer for a particular metric (like, for instance,
partition assignment) which will act as a good proxy for successful initialization. Currently,
we just check for the existence of a process with the PID, which is not a strong enough check,
causing the test to fail intermittently. 

This message was sent by Atlassian JIRA

View raw message