Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Fri, 2 Sep 2011 15:27:14 +0000 (UTC)
From: "Pavel Yaskevich (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <441874317.10781.1314977235001.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (CASSANDRA-1391) Allow Concurrent Schema
 Migrations
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096045#comment-13096045 ] 

Pavel Yaskevich commented on CASSANDRA-1391:
--------------------------------------------

bq. Could you clarify what you mean by "merging migration applied first, then all older migrations..."?

Take a look at the Migration.apply() starting from line 114 and Migration.tryMerge methods

if we detect that current migration is outdated and should be merged we do the following actions:

  - initialize isolated Schema from the point of migration's lastVersion (this sets isolated = true)
  - reload migration's system definition to reflect that isolated schema
  - call applyModels on the merging migration to apply it's schema changes
  - merge phrase:
    - read from SystemTable.Migrations all migrations that go after current
    - for each of those migrations:
       - replaces their schema with isolated (from merging migration) and reload system definition
       - call apply() method to re-write records in SystemTable.Migrations and SystemTable.Schema
  - after all migrations were applied we try to merge isolated schema with current system schema (Schema.instance)
  - flush system tables to persist changes

bq. It seems like a side effect of applying a migration is that it can apply other migrations. Does MigrationManager.applyMigrations() need to be updated because of this?

No because all modifications are done using isolated schema

bq. What does "isolated" indicate?

Isolated indicates that migration will be applied with isolated Schema so no real file operations are going to be made, such as snapshot, create of the keyspace directory, remove of the SSTable files etc.  
 
bq. Try to put things like flushSystemTables() in a separate patch (ok on the same ticket) to make reviewing the actual changes easier.

I see only one such a refactoring change, is it really worse splitting current patch?

bq. Would it be possible to create some unit tests for CFMD.diff()

CFMD.diff is used all over the place so if it was broken other tests would fail but if you think that this is necessary I can do that.

> Allow Concurrent Schema Migrations
> ----------------------------------
>
>                 Key: CASSANDRA-1391
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1391
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Stu Hood
>            Assignee: Pavel Yaskevich
>             Fix For: 1.0
>
>         Attachments: CASSANDRA-1391.patch
>
>
> CASSANDRA-1292 fixed multiple migrations started from the same node to properly queue themselves, but it is still possible for migrations initiated on different nodes to conflict and leave the cluster in a bad state. Since the system_add/drop/rename methods are accessible directly from the client API, they should be completely safe for concurrent use.
> It should be possible to allow for most types of concurrent migrations by converting the UUID schema ID into a VersionVectorClock (as provided by CASSANDRA-580).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira