Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Message-ID: <30775276.196901290178694397.JavaMail.jira@thor>
Date: Fri, 19 Nov 2010 09:58:14 -0500 (EST)
From: "Gary Dusbabek (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Subject: [jira] Commented: (CASSANDRA-1715) More schema migration race
 conditions
In-Reply-To: <5656367.44291288991565212.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933827#action_12933827 ] 

Gary Dusbabek commented on CASSANDRA-1715:
------------------------------------------

bq. right, this is what 0011 does. more accurately, it uses CSLM to avoid an explicit lock.
My argument was that it wasn't safe to call indexCfs.removeAllSSTables() without the flush lock, but it looks like SSTableTracker is properly synchronized to avoid any problems with flushing.  No problem here.

+1 I'll commit this shortly.

> More schema migration race conditions
> -------------------------------------
>
>                 Key: CASSANDRA-1715
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1715
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7 beta 1
>            Reporter: Jonathan Ellis
>            Assignee: Gary Dusbabek
>            Priority: Critical
>             Fix For: 0.7.0
>
>         Attachments: v3-0001-take-drop-off-CompactionManager.txt, v3-0002-compaction-lock.txt, v3-0003-migration-uses-locks.txt, v3-0004-handle-moved-dropped-CF-prior-to-pending-compaction-st.txt, v3-0005-CFS.reload-assumes-metadata-is-mutable.txt, v3-0006-replace-modifiable-CFM-members-with-private-fields-a.patch, v3-0007-updateColumnFamily-uses-reload-remove-unneccesary-stru.txt, v3-0008-perform-index-maintenance-outside-of-migration-locks-d.txt, v3-0009-use-avro-structures-inside-UpdateColumnFamily.txt, v3-0010-remove-unused-fields-in-DropColumnFamily-DropKeyspace.txt, v3-0011-make-addIndex-asynchronous-and-race-proof.patch, v3-0012-remove-locks-from-UpdateColumnFamily.patch
>
>
> Related to CASSANDRA-1631.
> This is still a bug with schema updates to an existing CF, since reloadCf is doing a unload/init cycle. So flushing + compaction is an issue there as well. Here is a stacktrace from during an index creation where it stubbed its toe on an incomplete sstable from an in-progress compaction (path names anonymized):
> {code}
> INFO [CompactionExecutor:1] 2010-11-02 16:31:00,553 CompactionManager.java (line 224) Compacting [org.apache.cassandra.io.sstable.SSTableReader(path='Standard1-e-6-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='Standard1-e-7-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='Standard1-e-8-Data.db'),org.apache.cassandra.io.sstable.SSTableReader(path='Standard1-e-9-Data.db')]
> ...
> ERROR [MigrationStage:1] 2010-11-02 16:31:10,939 ColumnFamilyStore.java (line 244) Corrupt sstable Standard1-tmp-e-10-<>=[Data.db, Index.db]; skipped
> java.io.EOFException
>         at org.apache.cassandra.utils.FBUtilities.skipShortByteArray(FBUtilities.java:308)
>         at org.apache.cassandra.io.sstable.SSTable.estimateRowsFromIndex(SSTable.java:231)
>         at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:286)
>         at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:202)
>         at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:235)
>         at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:443)
>         at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:431)
>         at org.apache.cassandra.db.Table.initCf(Table.java:335)
>         at org.apache.cassandra.db.Table.reloadCf(Table.java:343)
>         at org.apache.cassandra.db.migration.UpdateColumnFamily.applyModels(UpdateColumnFamily.java:89)
>         at org.apache.cassandra.db.migration.Migration.apply(Migration.java:158)
>         at org.apache.cassandra.thrift.CassandraServer$2.call(CassandraServer.java:672)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> ...
>  INFO [CompactionExecutor:1] 2010-11-02 16:31:31,970 CompactionManager.java (line 303) Compacted to Standard1-tmp-e-10-Data.db.  213,657,983 to 213,657,983 (~100% of original) bytes for 626,563 keys.  Time: 31,416ms.
> {code}
> There is also a race between schema modification and streaming.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.