Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Sat, 16 Mar 2013 15:36:12 +0000 (UTC)
From: "Omid Aladini (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12629730.1359479324152.452251.1363448172978@arcas>
In-Reply-To: <JIRA.12629730.1359479324152@arcas>
References: <JIRA.12629730.1359479324152@arcas>
Subject: [jira] [Comment Edited] (CASSANDRA-5195) Offline scrub does not
 migrate the directory structure on migration from 1.0.x to 1.1.x and causes
 the keyspace to disappear
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604295#comment-13604295 ] 

Omid Aladini edited comment on CASSANDRA-5195 at 3/16/13 3:36 PM:
------------------------------------------------------------------

I think upon loading the schema via OfflineScrubber, DefsTable.loadFromStorage migrates the old system tables to the new format. Therefore it drops the old ones but doesn't flush the commitlogs of new ones. OfflineScrubber exits and on the next start, schema _keyspaces, _columnfamilies and _columns CFs have no persisted sstables, and the commitlog gets replayed after Cassandra tries to load CF schemas, therefore finds none. This explains why after second restart, column families appear again (something force-flushes system CFs?).

A new patch (0001-Flush-newly-migrated-system-CFs.patch) is attached to fix the problem (although I'm not sure if there is a more proper fix for this or if there is a better place to put the foced flush.)
                
      was (Author: omid):
    I think upon loading the schema via OfflineScrubber, DefsTable.loadFromStorage migrates the old system tables to the new format. Therefore it drops the old ones but doesn't flush the commitlogs of new ones. OfflineScrubber exits and on the next start, schema_{keyspaces, columnfamilies, columns} have no persisted sstables, and the commitlog gets replayed after Cassandra tries to load CF schemas, therefore finds none. This explains why after second restart, column families appear again (something force-flushes system CFs?).

A new patch (0001-Flush-newly-migrated-system-CFs.patch) is attached to fix the problem (although I'm not sure if there is a more proper fix for this or if there is a better place to put the foced flush.)
                  
> Offline scrub does not migrate the directory structure on migration from 1.0.x to 1.1.x and causes the keyspace to disappear
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-5195
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5195
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.9
>            Reporter: Omid Aladini
>             Fix For: 1.1.11
>
>         Attachments: 0001-Flush-newly-migrated-system-CFs.patch, 5195.patch
>
>
> Due to CASSANDRA-4411, upon migration from 1.0.x to 1.1.x containing LCS-compacted sstables, an offline scrub should be run before Cassandra 1.1.x is started. But Cassandra 1.1.x uses a new directory structure (CASSANDRA-2749) that offline scrubber doesn't detect or try to migrate.
> How to reproduce:
> 1- Run cassandra 1.0.12.
> 2- Run stress tool, let Cassandra flush Keyspace1 or flush manually.
> 3- Stop cassandra 1.0.12
> 4- Run ./bin/sstablescrub Keyspace1 Standard1
>   which returns "Unknown keyspace/columnFamily Keyspace1.Standard1" and notice the data directory isn't migrated.
> 5- Run cassandra 1.1.9. Keyspace1 doesn't get loaded and Cassandra doesn't try to migrate the directory structure. Also commitlog entries get skipped: "Skipped XXXXX mutations from unknown (probably removed) CF with id 1000"
> Without the unsuccessful step 4, Cassandra 1.1.9 loads and migrates the Keyspace correctly.
>   

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira