cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Hobbs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9893) Fix upgrade tests from #9704 that are still failing
Date Fri, 07 Aug 2015 23:05:46 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662626#comment-14662626
] 

Tyler Hobbs commented on CASSANDRA-9893:
----------------------------------------

The three tests that appear to still be failing for me are: {{bug_5240_test}}, {{bug_5732_test}},
and {{edge_2i_on_complex_pk_test}}.  There may be other occasional failures that don't show
up on every run due to different token assignments (only the upgraded or non-upgraded node
may own a particularly partition).

The {{bug_5732_test}} failure is due to CASSANDRA-10006.

The new tests have a few useful environment variables that make it easier to debug problems.
 These are documented at the top of {{upgrade_tests/upgrade_base.py}}.  While working on these
tests, I normally used two cassandra directories: one for 3.0, and one for 2.1 with debug
logging.  For example, I might run a test like this:

{noformat}
UPGRADE_MODE=normal DEBUG=true CASSANDRA_DIR=/home/thobbs/cassandra OLD_CASSANDRA_DIR=/home/thobbs/cassandra2
nosetests  upgrade_tests/cql_tests.py:TestCQL.select_distinct_with_deletions_test
{noformat}

For 2.1 debug logging, I started this branch with an excessive amount of logging around command
and response serialization: https://github.com/thobbs/cassandra/tree/8099-backward-compat-2.1-debug.

The {{UPGRADE_MODE}} variable is useful for seeing exactly what commands or responses 2.1
or 3.0 expect.  For example, to see how 2.1 handles a particular query, you can do:

{noformat}
UPGRADE_MODE=none KEEP_LOGS=true DEBUG=true CASSANDRA_DIR=/home/thobbs/cassandra OLD_CASSANDRA_DIR=/home/thobbs/cassandra2
nosetests  upgrade_tests/cql_tests.py:TestCQL.select_distinct_with_deletions_test
{noformat}

Note the extra {{KEEP_LOGS=true}}, as the test should  pass without the upgrade step.

In some cases I've had problems with the new auth schema in 3.0.  This can be worked around
by doing something like this in {{upgrade_base.py}}:

{noformat}
node1.start(wait_for_binary_proto=True, jvm_args=['-Dcassandra.superuser_setup_delay_ms=1000000'])
{noformat}

However, I have not committed this, because I think this might cause legitimate problems for
users.  We probably either need to do CASSANDRA-9289 or investigate why reconnections are
still causing queries to be dropped.


> Fix upgrade tests from #9704 that are still failing
> ---------------------------------------------------
>
>                 Key: CASSANDRA-9893
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9893
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Sylvain Lebresne
>             Fix For: 3.0.0 rc1
>
>
> The first things to do on this ticket would be to commit Tyler's branch (https://github.com/thobbs/cassandra-dtest/tree/8099-backwards-compat)
to the dtests so cassci run them. I've had to do a few minor modifications to have them run
locally so someone which access to cassci should do it and make sure it runs properly.
> Once we have that, we should fix any test that isn't passing. I've ran the tests locally
and I had 8 failures. for 2 of them, it sounds plausible that they'll get fixed by the patch
of CASSANDRA-9775, though that's just a guess.  The rest where test that timeouted without
a particular error in the log, and running some of them individually, they passed.  So we'll
have to see if it's just my machine being overly slow when running them all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message