cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Huy Le (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-3819) Cannot restart server after making schema changes to composite CFs
Date Tue, 31 Jan 2012 17:42:14 GMT
Cannot restart server after making schema changes to composite CFs
------------------------------------------------------------------

                 Key: CASSANDRA-3819
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3819
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 1.0.7, 1.0.6
         Environment: Ubuntu 11.0.4
            Reporter: Huy Le


This JIRA is for issue discussed in this thread http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7150978.html.

We were using version 1.0.6.  We added new keyspace using built-in composite data type.  We
then decided to change the schema, specifically just the CF names, so we dropped the keyspace.
 We recreated the key space with different CF names in the key space.

There were a lot of uncommitted data in commit logs.  Data dated back before the original
key space was created.  When we restarted the server, the server failed when it read it the
commit logs, and the server stopped.  Here is snippet of the stack trace:

{code}
-3881-11e1-ac7f-12313d23ead3:true:4@1326223353559001,])}
DEBUG 18:02:01,057 Reading mutation at 66336992
DEBUG 18:02:01,058 replaying mutation for Springpad.696d6167652d7363616c65722d6d657461: {ColumnFamily(CassandraOrderedQueue
[0,eb321490-3881-11e1-ac7f-12313d23ead3:true:4@1326223353560000,])}
DEBUG 18:02:01,058 Reading mutation at 66337118
DEBUG 18:02:01,058 replaying mutation for Springpad.737072696e674d6f64656c44617461626173652d6d657461:
{ColumnFamily(CassandraOrderedQueue [0,80dc0cd0-3bc0-11e1-83a8-12313d23ead3:false:8@1326223386668000,])}
DEBUG 18:02:01,058 Reading mutation at 66337255
DEBUG 18:02:01,058 replaying mutation for system.38363233616337302d336263302d313165312d303030302d323366623834646463346633:
{ColumnFamily(Schema [Avro/Schema:false:2725@1326223386807,Backups:false:431@1326223386807,Springpad:false:10814@1326223386807,SpringpadGraph:false:2931@1326223386807,])}
DEBUG 18:02:01,059 Reading mutation at 66354352
DEBUG 18:02:01,059 replaying mutation for system.4d6967726174696f6e73204b6579: {ColumnFamily(Migrations
[8623ac70-3bc0-11e1-0000-23fb84ddc4f3:false:23728@1326223386812,])}
DEBUG 18:02:01,059 Reading mutation at 66378184
DEBUG 18:02:01,059 replaying mutation for system.4c617374204d6967726174696f6e: {ColumnFamily(Schema
[Last Migration:false:16@1326223386812,])}
DEBUG 18:02:01,059 Reading mutation at 66378302
 INFO 18:02:01,060 Finished reading /mnt/cassandra/commitlog/CommitLog-1325861435420.log
ERROR 18:02:01,061 Exception encountered during startup
java.lang.IllegalArgumentException
        at java.nio.Buffer.limit(Buffer.java:247)
        at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:57)
        at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:66)
        at org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:129)
        at org.apache.cassandra.db.Column.getString(Column.java:250)
        at org.apache.cassandra.db.marshal.AbstractType.getColumnsString(AbstractType.java:137)
        at org.apache.cassandra.db.ColumnFamily.toString(ColumnFamily.java:280)
        at org.apache.commons.lang.ObjectUtils.toString(ObjectUtils.java:241)
        at org.apache.commons.lang.StringUtils.join(StringUtils.java:3073)
        at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:301)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:172)
        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:215)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:356)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:107)
Exception encountered during startup: null 
{code}

Sample original CF schema:
{code}
create column family InEdges
  with column_type = 'Standard'
  and comparator = 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
  and default_validation_class = 'BytesType'
  and key_validation_class = 'UTF8Type'
  and rows_cached = 0.0
  and row_cache_save_period = 0
  and row_cache_keys_to_save = 0
  and keys_cached = 0.0
  and key_cache_save_period = 0
  and read_repair_chance = 0.0
  and gc_grace = 0
  and min_compaction_threshold = 4
  and max_compaction_threshold = 32
  and replicate_on_write = true
  and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
  and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
{code}

And the updated schema (just CF name change):
{code}
create column family NewInEdges
  with column_type = 'Standard'
  and comparator = 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
  and default_validation_class = 'BytesType'
  and key_validation_class = 'UTF8Type'
  and rows_cached = 0.0
  and row_cache_save_period = 0
  and row_cache_keys_to_save = 0
  and keys_cached = 0.0
  and key_cache_save_period = 0
  and read_repair_chance = 0.0
  and gc_grace = 0
  and min_compaction_threshold = 4
  and max_compaction_threshold = 32
  and replicate_on_write = true
  and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
  and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
{code}

This issue happens every time composite type is used and workaround can be found at (http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7226863.html).




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message