cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel van 't Oever (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5025) Schema push/pull race
Date Fri, 06 Jan 2017 10:14:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804212#comment-15804212
] 

Daniel van 't Oever commented on CASSANDRA-5025:
------------------------------------------------

I experience similar problems in a one node cluster. When I start 3 applications at the same
time, they all try to migrate the cassandra schema (but will wait for each other using a locking
table). However, they will check for this lock table using a CREATE TABLE IF NOT EXISTS

Cassandra driver 2.1.10.1

{noformat}
2017-01-03 09:57:22,372 · WARN · cluster2-nio-worker-1 · com.datastax.driver.core.RequestHandler
· /127.0.0.1:9042 replied with server error (java.lang.RuntimeException: java.util.concurrent.ExecutionException:
org.apache.cassandra.exceptions.ConfigurationException: Column family ID mismatch (found a1c87c40-d192-11e6-a126-1d2c09c16740;
expected a1c56f00-d192-11e6-a126-1d2c09c16740)), defuncting connection. ·  ·  ·  ·
2017-01-03 09:57:22,394 · ERROR · main · com.contrastsecurity.cassandra.migration.action.Migrate
· Migration of keyspace ces2 to version 0.1.0.1 failed! Please restore backups and roll back
database and code! ·  ·  ·  ·
2017-01-03 09:57:24,652 · ERROR · main · nl.mypackage.CassandraMigrationService · Error
during migration ·  ·  ·  ·
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query
failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout
while trying to acquire available connection (you may want to increase the driver number of
per-host connections)))
        at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
        at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
        at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:217)
       at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:54)
        at com.contrastsecurity.cassandra.migration.dao.SchemaVersionDAO.tablesExist(SchemaVersionDAO.java:88)
{noformat}

What is the recommended way to perform schema migrations in a Cassandra cluster?

> Schema push/pull race
> ---------------------
>
>                 Key: CASSANDRA-5025
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5025
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.0
>            Reporter: Jonathan Ellis
>            Assignee: Jonathan Ellis
>            Priority: Minor
>             Fix For: 1.1.8
>
>         Attachments: 5025-v2.txt, 5025-v3.txt, 5025-v4.txt, 5025-v5.txt, 5025.txt
>
>
> When a schema change is made, the coordinator pushes the delta to the other nodes in
the cluster.  This is more efficient than sending the entire schema.  But the coordinator
also announces the new schema version, so the other nodes' reception of the new version races
with processing the delta, and usually seeing the new schema wins.  So the other nodes also
issue a pull to the coordinator for the entire schema.
> Thus, schema changes tend to become O(n) in the number of KS and CF present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message