incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Blair Zajac <bl...@orcaware.com>
Subject Cassandra won't restart : 7365....6c73 is not defined as a collection
Date Fri, 03 May 2013 02:36:50 GMT
Hello,

I'm running a 3-node development cluster on OpenStack VMs and recently 
updated to DataStax's 1.2.4 debs on Ubuntu Raring after which the 
cluster was fine.  I shut it down for a few days and after getting back 
to Cassandra today and booting the VMs, Cassandra is unable to start. 
Below is the output from output.log from one of the nodes.  None of the 
Cassandra nodes can start.

The deployment is pretty simple, two test keyspaces with a few column 
families in each keyspace.  I am doing a lot of keyspace and column 
family deletions as I'm testing some db style migration code to 
auto-setup a schema.

Any suggestions?

Blair

  INFO 19:24:09,780 Logging initialized
  INFO 19:24:09,790 JVM vendor/version: Java HotSpot(TM) 64-Bit Server 
VM/1.7.0_21
  INFO 19:24:09,791 Heap size: 880803840/880803840
  INFO 19:24:09,791 Classpath: 
/usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.6.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/guava-13.0.1.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.7.0.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/lz4-1.1.0.jar:/usr/share/cassandra/lib/metrics-core-2.0.3.jar:/usr/share/cassandra/lib/netty-3.5.9.Final.jar:/usr/share/cassand!
 ra/lib/ser
vlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.4.1.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/apache-cassandra-1.2.4.jar:/usr/share/cassandra/apache-cassandra-thrift-1.2.4.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar
  INFO 19:24:09,987 JNA mlockall successful
  INFO 19:24:10,001 Loading settings from file:/etc/cassandra/cassandra.yaml
  INFO 19:24:10,371 Data files directories: [/var/lib/cassandra/data]
  INFO 19:24:10,372 Commit log directory: /var/lib/cassandra/commitlog
  INFO 19:24:10,372 DiskAccessMode 'auto' determined to be mmap, 
indexAccessMode is mmap
  INFO 19:24:10,372 disk_failure_policy is stop
  INFO 19:24:10,377 Global memtable threshold is enabled at 280MB
  INFO 19:24:10,474 Not using multi-threaded compaction
  INFO 19:24:10,816 Initializing key cache with capacity of 42 MBs.
  INFO 19:24:10,822 Scheduling key cache save to each 14400 seconds 
(going to save all keys).
  INFO 19:24:10,823 Initializing row cache with capacity of 0 MBs and 
provider org.apache.cassandra.cache.SerializingCacheProvider
  INFO 19:24:10,827 Scheduling row cache save to each 0 seconds (going 
to save all keys).
  INFO 19:24:10,958 Opening 
/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ib-165 
(35 bytes)
  INFO 19:24:10,989 Opening 
/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ib-166 
(168 bytes)
  INFO 19:24:10,991 Opening 
/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ib-164 
(346 bytes)
  INFO 19:24:10,999 reading saved cache 
/var/lib/cassandra/saved_caches/system-schema_keyspaces-KeyCache-b.db
  INFO 19:24:11,018 Opening 
/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ib-461 
(6562 bytes)
  INFO 19:24:11,024 reading saved cache 
/var/lib/cassandra/saved_caches/system-schema_columnfamilies-KeyCache-b.db
  INFO 19:24:11,031 Opening 
/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ib-394 
(465 bytes)
  INFO 19:24:11,032 Opening 
/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ib-395 
(244 bytes)
  INFO 19:24:11,033 Opening 
/var/lib/cassandra/data/system/schema_columns/system-schema_columns-ib-393 
(3025 bytes)
  INFO 19:24:11,035 reading saved cache 
/var/lib/cassandra/saved_caches/system-schema_columns-KeyCache-b.db
  INFO 19:24:11,048 Opening 
/var/lib/cassandra/data/system/peers/system-peers-ib-16 (295 bytes)
  INFO 19:24:11,052 Opening 
/var/lib/cassandra/data/system/peers/system-peers-ib-15 (309 bytes)
  INFO 19:24:11,057 Opening 
/var/lib/cassandra/data/system/peers/system-peers-ib-14 (10840 bytes)
  INFO 19:24:11,074 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-58 (97 bytes)
  INFO 19:24:11,076 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-57 (97 bytes)
  INFO 19:24:11,082 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-61 (150 bytes)
  INFO 19:24:11,086 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-56 (148 bytes)
  INFO 19:24:11,087 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-60 (97 bytes)
  INFO 19:24:11,089 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-53 (5846 bytes)
  INFO 19:24:11,091 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-55 (5410 bytes)
  INFO 19:24:11,094 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-59 (97 bytes)
  INFO 19:24:11,095 Opening 
/var/lib/cassandra/data/system/local/system-local-ib-54 (120 bytes)
  INFO 19:24:11,097 reading saved cache 
/var/lib/cassandra/saved_caches/system-local-KeyCache-b.db
  INFO 19:24:11,666 Opening 
/var/lib/cassandra/data/system_auth/users/system_auth-users-ib-1 (64 bytes)
  INFO 19:24:11,708 completed pre-loading (17 keys) key cache.
  INFO 19:24:11,732 Replaying 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867616.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867617.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867618.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867619.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867620.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867621.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867622.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867623.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867624.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867625.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867626.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867627.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367547324162.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367547728748.log, 
/var/lib/cassandra/commitlog/CommitLog-2-1367547828579.log
  INFO 19:24:11,783 Replaying 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867616.log
  INFO 19:24:11,870 Finished reading 
/var/lib/cassandra/commitlog/CommitLog-2-1367034867616.log
java.lang.RuntimeException: 7365636f6e646172795f656d61696c73 is not 
defined as a collection
	at 
org.apache.cassandra.db.marshal.ColumnToCollectionType.compareCollectionMembers(ColumnToCollectionType.java:69)
	at 
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:81)
	at 
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31)
	at java.util.TreeMap.put(TreeMap.java:545)
	at 
org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:102)
	at 
org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:88)
	at 
org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
	at 
org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:104)
	at 
org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:101)
	at 
org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:376)
	at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:203)
	at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:146)
	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:126)
	at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:279)
	at 
org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:356)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at 
org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:212)
Cannot load daemon

Mime
View raw message