Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Thu, 22 Sep 2011 22:23:28 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <1905277411.3956.1316730208381.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <302942565.35384.1316192409452.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (CASSANDRA-3218) Failure reading a
 erroneous/spurious AutoSavingCache file can result in a failed application
 of a migration, which can prevent a node from reaching schema agreement.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13112966#comment-13112966 ] 

Hudson commented on CASSANDRA-3218:
-----------------------------------

Integrated in Cassandra-0.8 #337 (See [https://builds.apache.org/job/Cassandra-0.8/337/])
    Don't allow any cache loading exceptions to halt startup
patch by Eldon Stegall; reviewed by jbellis for CASSANDRA-3218

jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1174392
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cache/AutoSavingCache.java


> Failure reading a erroneous/spurious AutoSavingCache file can result in a failed application of a migration, which can prevent a node from reaching schema agreement.
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3218
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3218
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.8.0
>            Reporter: Eldon Stegall
>            Assignee: Eldon Stegall
>            Priority: Minor
>             Fix For: 0.8.7, 1.0.0
>
>         Attachments: CASSANDRA-3218-autosave_exception_handling-v2.patch, CASSANDRA-3218-autosave_exception_handling.patch
>
>
> Failure reading a erroneous/spurious AutoSavingCache file can result in a failed application of a migration, which can prevent a node from reaching schema agreement. This is distinctly possible when a machine loses it's data partition, and attempts to recover the schema upon restart, and so has to apply all the migrations. The initial stack traces look like this:
> Add column family: org.apache.cassandra.config.CFMetaData@38bcfee6[cfId=1000,ksName=someks,cfName=somecf,cfType=Standard,comparat
> or=org.apache.cassandra.db.marshal.UTF8Type,subcolumncomparator=<null>, ...
> Followed by:
> ERROR 00:56:47,974 Fatal exception in thread Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.lang.NegativeArraySizeException
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.NegativeArraySizeException
>         at org.apache.cassandra.cache.AutoSavingCache.readSaved(AutoSavingCache.java:130)
>         at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:273)
>         at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:465)
>         at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:435)
>         at org.apache.cassandra.db.Table.initCf(Table.java:369)
>         at org.apache.cassandra.db.migration.AddColumnFamily.applyModels(AddColumnFamily.java:93)
>         at org.apache.cassandra.db.migration.Migration.apply(Migration.java:153)
>         at org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:73)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         ... 6 more
> Ultimately, attempted changes to this keyspace/cf will fail like this:
> ERROR 13:07:51,006 Fatal exception in thread Thread[MigrationStage:1,5,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException: Unknown CF 1000
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException: Unknown CF 1000
>         at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:155)
>         at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:148)
>         at org.apache.cassandra.db.migration.DropKeyspace.applyModels(DropKeyspace.java:63)
>         at org.apache.cassandra.db.migration.Migration.apply(Migration.java:153)
>         at org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:73)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         ... 6 more

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira