cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oliver Lockwood (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11288) Schema agreement appears to be false positive following a DROP TABLE command
Date Wed, 02 Mar 2016 09:29:18 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175324#comment-15175324
] 

Oliver Lockwood commented on CASSANDRA-11288:
---------------------------------------------

I'm going to bump the severity to Major since I've got fairly reliable reproduction of this
issue and it's manifested in 4-node environments as well as simpler 2-node ones.

See the log output below for a fuller example.  What's interesting is that the "agreed" schema
AFTER the DROP TABLE command has been issued is exactly the same version ({c264a58b-a3a4-39f8-84ae-c0960b7e8b4c})
as that at the end of the previous DDL change.  So it looks like the nodes agree that nothing
has changed yet, even though the EVENT DROPPED TABLE has come through.

{quote}
2016-03-02 09:11:13 DEBUG SchemaChanger:61 - Executing cql statement ALTER TABLE table_8 WITH
compaction = { 'class' :  'LeveledCompactionStrategy'  };
2016-03-02 09:11:13 DEBUG Cluster:2256 - Received event EVENT UPDATED TABLE testkeyspace.table_8,
scheduling delivery
2016-03-02 09:11:13 DEBUG Cluster:2199 - Refreshing schema for testkeyspace.table_8 (TABLE)
2016-03-02 09:11:13 DEBUG ControlConnection:676 - Checking for schema agreement: versions
are [d138c607-135d-3081-99ec-350661791061, c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:14 DEBUG ControlConnection:676 - Checking for schema agreement: versions
are [c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:15 DEBUG ControlConnection:294 - [Control connection] Refreshing schema for
testkeyspace.table_8 (TABLE)
2016-03-02 09:11:15 DEBUG SchemaChanger:61 - Executing cql statement DROP TABLE table_1;
2016-03-02 09:11:15 DEBUG Cluster:2256 - Received event EVENT DROPPED TABLE testkeyspace.table_1,
scheduling delivery
2016-03-02 09:11:15 DEBUG ControlConnection:676 - Checking for schema agreement: versions
are [c264a58b-a3a4-39f8-84ae-c0960b7e8b4c]
2016-03-02 09:11:15 DEBUG SchemaChanger:61 - Executing cql statement CREATE TABLE table_1
(party_id text primary key, profile text) WITH compaction = { 'class' :  'LeveledCompactionStrategy'
};
Exception in thread "main" com.datastax.driver.core.exceptions.AlreadyExistsException: Table
testkeyspace.table_1 already exists
	at com.datastax.driver.core.exceptions.AlreadyExistsException.copy(AlreadyExistsException.java:111)
	at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
	at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:214)
	at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52){quote}

> Schema agreement appears to be false positive following a DROP TABLE command
> ----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-11288
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11288
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: Cassandra 2.0.14.439 (DSE 4.6.7)
> 2 nodes OR 4 nodes
> Connecting with Datastax Java driver 2.1.8
>            Reporter: Oliver Lockwood
>            Priority: Minor
>
> As part of a schema migration operation, our application is calling the following operations
on the Java driver consecutively:
> {quote}
> session.execute("DROP TABLE table_name");
> session.execute("CREATE TABLE table_name (...)");
> {quote}
> The second of these sometimes fails with a {{DriverException}} whose message is "Table
keyspace.table_name already exists".
> In the schema migration operation, there's 4 of these drop/create pairings and, although
it's random which exact one fails, we've never managed to get further than the third operation
in approximately 10 attempts - so there's a reasonably high proportion of failure.
> I don't believe this is a driver issue because the driver is checking for schema agreement
(as per https://github.com/datastax/java-driver/blob/2.1/driver-core/src/main/java/com/datastax/driver/core/ControlConnection.java#L701)
and we are seeing a log message to that effect.
> {quote}
> c.d.d.c.ControlConnection - [] [] [] [] [] [] [] [] Checking for schema agreement: versions
are [02bce936-fddd-3bef-bb54-124d31bede57]
> {quote}
> This log message appears in between our own logs which say "Executing statement DROP
TABLE..." and "Executing statement CREATE TABLE...", so we can be reasonably sure this log
message refers to the DROP operation being viewed as "in agreement".
> Could this be a bug in the Cassandra server erroneously reporting that the schemas are
in agreement across the 2 nodes when, in fact, they are not?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message