cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Huy Le (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3819) Cannot restart server after making schema changes to composite CFs
Date Sat, 04 Feb 2012 16:48:53 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200475#comment-13200475
] 

Huy Le commented on CASSANDRA-3819:
-----------------------------------

I did not try to reproduce it.  Another user mentioned in this thread http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7226863.html
that it happened every time.

As for me, I have 3 environments.  This issue happened on an environment where there where
uncommitted logs accumulated before the schema changes. Two other environments have committed
logs fully flushed before the restart did not suffer this issue.  In my environments, there
were data inserted into the node before and after schema changes.


                
> Cannot restart server after making schema changes to composite CFs
> ------------------------------------------------------------------
>
>                 Key: CASSANDRA-3819
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3819
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.6, 1.0.7
>         Environment: Ubuntu 11.0.4
>            Reporter: Huy Le
>            Assignee: Sylvain Lebresne
>             Fix For: 1.0.8
>
>
> This JIRA is for issue discussed in this thread http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7150978.html.
> We were using version 1.0.6.  We added new keyspace using built-in composite data type.
 We then decided to change the schema, specifically just the CF names, so we dropped the keyspace.
 We recreated the key space with different CF names in the key space.
> There were a lot of uncommitted data in commit logs.  Data dated back before the original
key space was created.  When we restarted the server, the server failed when it read it the
commit logs, and the server stopped.  Here is snippet of the stack trace:
> {code}
> -3881-11e1-ac7f-12313d23ead3:true:4@1326223353559001,])}
> DEBUG 18:02:01,057 Reading mutation at 66336992
> DEBUG 18:02:01,058 replaying mutation for Springpad.696d6167652d7363616c65722d6d657461:
{ColumnFamily(CassandraOrderedQueue [0,eb321490-3881-11e1-ac7f-12313d23ead3:true:4@1326223353560000,])}
> DEBUG 18:02:01,058 Reading mutation at 66337118
> DEBUG 18:02:01,058 replaying mutation for Springpad.737072696e674d6f64656c44617461626173652d6d657461:
{ColumnFamily(CassandraOrderedQueue [0,80dc0cd0-3bc0-11e1-83a8-12313d23ead3:false:8@1326223386668000,])}
> DEBUG 18:02:01,058 Reading mutation at 66337255
> DEBUG 18:02:01,058 replaying mutation for system.38363233616337302d336263302d313165312d303030302d323366623834646463346633:
{ColumnFamily(Schema [Avro/Schema:false:2725@1326223386807,Backups:false:431@1326223386807,Springpad:false:10814@1326223386807,SpringpadGraph:false:2931@1326223386807,])}
> DEBUG 18:02:01,059 Reading mutation at 66354352
> DEBUG 18:02:01,059 replaying mutation for system.4d6967726174696f6e73204b6579: {ColumnFamily(Migrations
[8623ac70-3bc0-11e1-0000-23fb84ddc4f3:false:23728@1326223386812,])}
> DEBUG 18:02:01,059 Reading mutation at 66378184
> DEBUG 18:02:01,059 replaying mutation for system.4c617374204d6967726174696f6e: {ColumnFamily(Schema
[Last Migration:false:16@1326223386812,])}
> DEBUG 18:02:01,059 Reading mutation at 66378302
>  INFO 18:02:01,060 Finished reading /mnt/cassandra/commitlog/CommitLog-1325861435420.log
> ERROR 18:02:01,061 Exception encountered during startup
> java.lang.IllegalArgumentException
>         at java.nio.Buffer.limit(Buffer.java:247)
>         at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:57)
>         at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:66)
>         at org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:129)
>         at org.apache.cassandra.db.Column.getString(Column.java:250)
>         at org.apache.cassandra.db.marshal.AbstractType.getColumnsString(AbstractType.java:137)
>         at org.apache.cassandra.db.ColumnFamily.toString(ColumnFamily.java:280)
>         at org.apache.commons.lang.ObjectUtils.toString(ObjectUtils.java:241)
>         at org.apache.commons.lang.StringUtils.join(StringUtils.java:3073)
>         at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133)
>         at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:301)
>         at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:172)
>         at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:215)
>         at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:356)
>         at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:107)
> Exception encountered during startup: null 
> {code}
> Sample original CF schema:
> {code}
> create column family InEdges
>   with column_type = 'Standard'
>   and comparator = 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
>   and default_validation_class = 'BytesType'
>   and key_validation_class = 'UTF8Type'
>   and rows_cached = 0.0
>   and row_cache_save_period = 0
>   and row_cache_keys_to_save = 0
>   and keys_cached = 0.0
>   and key_cache_save_period = 0
>   and read_repair_chance = 0.0
>   and gc_grace = 0
>   and min_compaction_threshold = 4
>   and max_compaction_threshold = 32
>   and replicate_on_write = true
>   and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
>   and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
> {code}
> And the updated schema (just CF name change):
> {code}
> create column family NewInEdges
>   with column_type = 'Standard'
>   and comparator = 'CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)'
>   and default_validation_class = 'BytesType'
>   and key_validation_class = 'UTF8Type'
>   and rows_cached = 0.0
>   and row_cache_save_period = 0
>   and row_cache_keys_to_save = 0
>   and keys_cached = 0.0
>   and key_cache_save_period = 0
>   and read_repair_chance = 0.0
>   and gc_grace = 0
>   and min_compaction_threshold = 4
>   and max_compaction_threshold = 32
>   and replicate_on_write = true
>   and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
>   and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
> {code}
> This issue happens every time composite type is used and workaround can be found at (http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cannot-start-cassandra-node-anymore-tp7150978p7226863.html).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message