kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5608) System test failure due to timeout starting Jmx tool
Date Wed, 19 Jul 2017 00:43:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092414#comment-16092414
] 

Ewen Cheslack-Postava commented on KAFKA-5608:
----------------------------------------------

Further adding to the confusion, I've found a test ConnectMysqlTest.test_query_modes.query_mode=table.incremental_mode=timestamp
which has this in the jmx_tool.err.log:

{quote}
Trying to connect to JMX url: service:jmx:rmi:///jndi/rmi://127.0.0.1:9192/jmxrmi.
Trying to connect to JMX url: service:jmx:rmi:///jndi/rmi://127.0.0.1:9192/jmxrmi.
{quote}

But based on the code https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/JmxTool.scala#L98-L110
it doesn't seem that it should be possible to see that message twice without also seeing an
error message...

> System test failure due to timeout starting Jmx tool
> ----------------------------------------------------
>
>                 Key: KAFKA-5608
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5608
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Gustafson
>            Assignee: Ewen Cheslack-Postava
>
> Began seeing this in some failing system tests:
> {code}
> [INFO  - 2017-07-18 14:25:55,375 - background_thread - _protected_worker - lineno:39]:
Traceback (most recent call last):
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/services/background_thread.py",
line 35, in _protected_worker
>     self._worker(idx, node)
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/tests/kafkatest/services/console_consumer.py",
line 261, in _worker
>     self.start_jmx_tool(idx, node)
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/tests/kafkatest/services/monitor/jmx.py",
line 73, in start_jmx_tool
>     wait_until(lambda: self._jmx_has_output(node), timeout_sec=10, backoff_sec=.5, err_msg="%s:
Jmx tool took too long to start" % node.account)
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/utils/util.py",
line 36, in wait_until
>     raise TimeoutError(err_msg)
> TimeoutError: ubuntu@worker7: Jmx tool took too long to start
> {code}
> This is immediately followed by a consumer timeout in the failing cases:
> {code}
> [INFO  - 2017-07-18 14:26:46,907 - runner_client - log - lineno:221]: RunnerClient: kafkatest.tests.core.security_rolling_upgrade_test.TestSecurityRollingUpgrade.test_rolling_upgrade_phase_two.broker_protocol=SASL_SSL.client_protocol=SASL_SSL:
FAIL: Consumer failed to consume messages for 60s.
> Traceback (most recent call last):
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py",
line 123, in run
>     data = self.run_test()
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py",
line 176, in run_test
>     return self.test_context.function(self.test)
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/mark/_mark.py",
line 321, in wrapper
>     return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py",
line 148, in test_rolling_upgrade_phase_two
>     self.run_produce_consume_validate(self.roll_in_secured_settings, client_protocol,
broker_protocol)
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/tests/kafkatest/tests/produce_consume_validate.py",
line 106, in run_produce_consume_validate
>     self.start_producer_and_consumer()
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/tests/kafkatest/tests/produce_consume_validate.py",
line 79, in start_producer_and_consumer
>     self.consumer_start_timeout_sec)
>   File "/home/jenkins/workspace/system-test-kafka-0.11.0/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/utils/util.py",
line 36, in wait_until
>     raise TimeoutError(err_msg)
> TimeoutError: Consumer failed to consume messages for 60s.
> {code}
> There does not appear to be anything wrong with the consumer in the logs, so the timeout
seems to be caused by the Jmx tool timeout.
> Possibly due to https://github.com/apache/kafka/pull/3447/files?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message