Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Sat, 7 Jan 2017 01:49:58 +0000 (UTC)
From: "Apurva Mehta (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.13029284.1482189172000.675114.1483753798472@Atlassian.JIRA>
In-Reply-To: <JIRA.13029284.1482189172000@Atlassian.JIRA>
References: <JIRA.13029284.1482189172000@Atlassian.JIRA> <JIRA.13029284.1482189172455@arcas>
Subject: [jira] [Commented] (KAFKA-4558) throttling_test fails if the
 producer starts too fast.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sat, 07 Jan 2017 01:50:00 -0000


    [ https://issues.apache.org/jira/browse/KAFKA-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806466#comment-15806466 ] 

Apurva Mehta commented on KAFKA-4558:
-------------------------------------

I dug a bit further into the `JmxMixin`, and it seems that all we have to do in the console consumer is something like the following:

{code}
def has_partitions_assigned(self): 
  # we run at init jmx_object_names += ['partitions-assigned']
  self.read_jmx_output(idx, node)
  return self.average_jmx_value['partitions_assigned'] > 0
{code}

Then the current `wait_until` in `ProduceConsumeValidate.start_producer_and_consumer` should just wait until `has_partitions_assigned` returns true. Does this seem reasonable? Or have I misunderstood how this is supposed to work?   

> throttling_test fails if the producer starts too fast.
> ------------------------------------------------------
>
>                 Key: KAFKA-4558
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4558
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Apurva Mehta
>            Assignee: Apurva Mehta
>
> As described in https://issues.apache.org/jira/browse/KAFKA-4526, the throttling test will fail if the producer in the produce-consume-validate loop starts up before the consumer is fully initialized.
> We need to block the start of the producer until the consumer is ready to go. 
> The current plan is to poll the consumer for a particular metric (like, for instance, partition assignment) which will act as a good proxy for successful initialization. Currently, we just check for the existence of a process with the PID, which is not a strong enough check, causing the test to fail intermittently. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)