kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apurva Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4558) throttling_test fails if the producer starts too fast.
Date Sat, 07 Jan 2017 01:49:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806466#comment-15806466

Apurva Mehta commented on KAFKA-4558:

I dug a bit further into the `JmxMixin`, and it seems that all we have to do in the console
consumer is something like the following:

def has_partitions_assigned(self): 
  # we run at init jmx_object_names += ['partitions-assigned']
  self.read_jmx_output(idx, node)
  return self.average_jmx_value['partitions_assigned'] > 0

Then the current `wait_until` in `ProduceConsumeValidate.start_producer_and_consumer` should
just wait until `has_partitions_assigned` returns true. Does this seem reasonable? Or have
I misunderstood how this is supposed to work?   

> throttling_test fails if the producer starts too fast.
> ------------------------------------------------------
>                 Key: KAFKA-4558
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4558
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Apurva Mehta
>            Assignee: Apurva Mehta
> As described in https://issues.apache.org/jira/browse/KAFKA-4526, the throttling test
will fail if the producer in the produce-consume-validate loop starts up before the consumer
is fully initialized.
> We need to block the start of the producer until the consumer is ready to go. 
> The current plan is to poll the consumer for a particular metric (like, for instance,
partition assignment) which will act as a good proxy for successful initialization. Currently,
we just check for the existence of a process with the PID, which is not a strong enough check,
causing the test to fail intermittently. 

This message was sent by Atlassian JIRA

View raw message