Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Tue, 8 Dec 2015 18:21:11 +0000 (UTC)
From: "Greg Bestland (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12914164.1447865126000.303919.1449598871067@Atlassian.JIRA>
In-Reply-To: <JIRA.12914164.1447865126000@Atlassian.JIRA>
References: <JIRA.12914164.1447865126000@Atlassian.JIRA>
 <JIRA.12914164.1447865126028@arcas>
Subject: [jira] [Comment Edited] (CASSANDRA-10730) periodic timeout errors
 in dtest
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-10730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047187#comment-15047187 ] 

Greg Bestland edited comment on CASSANDRA-10730 at 12/8/15 6:20 PM:
--------------------------------------------------------------------

Adam Holmbeg,

Not that specific issue no. We have seem similar timeout issues when the disk on the instances we were using was extremely slow.

I have seen quite a few intermittent non specific timeout issues that manifest themselves in the following way.

{code}
 File "/home/jenkins/.pyenv/versions/2.7.10/lib/python2.7/site-packages/unittest2/case.py", line 67, in testPartExecutor
    yield
  File "/home/jenkins/.pyenv/versions/2.7.10/lib/python2.7/site-packages/unittest2/case.py", line 625, in run
    testMethod()
  File "/home/jenkins/workspace/datastax.python-driver.master/f50f1487/tests/integration/standard/test_metadata.py", line 430, in test_refresh_schema_metadata
    self.session.execute("CREATE TABLE {0}.{1} (a int PRIMARY KEY, b text)".format(self.keyspace_name, table_name))
  File "/home/jenkins/workspace/datastax.python-driver.master/f50f1487/cassandra/cluster.py", line 1569, in execute
    return self.execute_async(query, parameters, trace, custom_payload, timeout).result()
  File "/home/jenkins/workspace/datastax.python-driver.master/f50f1487/cassandra/cluster.py", line 3122, in result
    raise self._final_exception
"errors={}, last_host=127.0.0.3\n-------------------- >> begin captured logging << --------------------\n[DEBUG] 2015-11-25 
{code}

These usually occur around table creates,drops, or keyspace create/dropping. I cannot reproduce them locally. 


was (Author: greg.bestland):
Adam Holmbeg,

Not that specific issue no. We have seem similar timeout issues when the disk on the instances we were using was extremely slow.

I have seen quite a few intermittent non specific timeout issues that manifest themselves in the following way.

{code}
 File "/home/jenkins/.pyenv/versions/2.7.10/lib/python2.7/site-packages/unittest2/case.py", line 67, in testPartExecutor
    yield
  File "/home/jenkins/.pyenv/versions/2.7.10/lib/python2.7/site-packages/unittest2/case.py", line 625, in run
    testMethod()
  File "/home/jenkins/workspace/datastax.python-driver.master/f50f1487/tests/integration/standard/test_metadata.py", line 430, in test_refresh_schema_metadata
    self.session.execute("CREATE TABLE {0}.{1} (a int PRIMARY KEY, b text)".format(self.keyspace_name, table_name))
  File "/home/jenkins/workspace/datastax.python-driver.master/f50f1487/cassandra/cluster.py", line 1569, in execute
    return self.execute_async(query, parameters, trace, custom_payload, timeout).result()
  File "/home/jenkins/workspace/datastax.python-driver.master/f50f1487/cassandra/cluster.py", line 3122, in result
    raise self._final_exception
"errors={}, last_host=127.0.0.3\n-------------------- >> begin captured logging << --------------------\n[DEBUG] 2015-11-25 
{/code}

These usually occur around table creates,drops, or keyspace create/dropping. I cannot reproduce them locally. 


> periodic timeout errors in dtest
> --------------------------------
>
>                 Key: CASSANDRA-10730
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10730
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jim Witschey
>            Assignee: Jim Witschey
>
> Dtests often fail with connection timeout errors. For example:
> http://cassci.datastax.com/job/cassandra-3.1_dtest/lastCompletedBuild/testReport/upgrade_tests.cql_tests/TestCQLNodes3RF3/deletion_test/
> {code}
> ('Unable to connect to any servers', {'127.0.0.1': OperationTimedOut('errors=Timed out creating connection (10 seconds), last_host=None',)})
> {code}
> We've merged a PR to increase timeouts:
> https://github.com/riptano/cassandra-dtest/pull/663
> It doesn't look like this has improved things:
> http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/363/testReport/
> Next steps here are
> * to scrape Jenkins history to see if and how the number of tests failing this way has increased (it feels like it has). From there we can bisect over the dtests, ccm, or C*, depending on what looks like the source of the problem.
> * to better instrument the dtest/ccm/C* startup process to see why the nodes start but don't successfully make the CQL port available.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)