incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Morton <aa...@thelastpickle.com>
Subject Re: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column name length 0
Date Tue, 01 Feb 2011 20:48:09 GMT
I think the problem here is your test.

The CassandraServer class calls validateColumns() and will not let zero length columns be
inserted. The call would not get to the storage proxy.

Aaron

On 2/02/2011, at 2:27 AM, Mikael Wikblom <mikael.wikblom@sitevision.se> wrote:

> Hi,
> 
> I'm getting an exception when doing many writes to a column with an
> empty column name (i.e. empty byte buffer). The exception occurs during
> compactation. It is reproducable given enough writes (around 1300000).
> Is this a known issue? After the problem occurs cassandra will no longer
> start up. It starts up again after I've manually deleted the commit log
> files.
> 
> I have failed to reproduce the problem with a non empty byte array.
> 
> I'm using cassandra 0.7
> 
> Is there a way to recover from such errors without loosing the entries
> in the commit log?
> 
> Regards
> 
> Mikael Wikblom
> 
> 
> 
> Here is my unit test, the exception and the yaml. I also provided the
> stacktrace from the following startup exception
> 
> ===unit test==============================
> 
> @Test
>   public void testInvalidColumnNames()
>         throws Exception
>   {
>      final Charset CHARSET = Charset.forName("UTF-8");
> 
>      int count = 1300000;
>      for (int i = 0; i < count; i++)
>      {
>         List<RowMutation> mutations = new ArrayList<RowMutation>();
>         RowMutation m = new RowMutation("Test",
> ByteBuffer.wrap(("rowId").getBytes(CHARSET)));
>         // QueryPath qp = new QueryPath("TestCassandra", null,
> ByteBuffer.wrap(("test" + i).getBytes(CHARSET)));
>         QueryPath qp = new QueryPath("TestCassandra", null,
> FBUtilities.EMPTY_BYTE_BUFFER);
> 
>         m.add(qp, ByteBuffer.wrap("test".getBytes(CHARSET)),
> System.currentTimeMillis());
>         mutations.add(m);
> 
>         StorageProxy.mutate(mutations, ConsistencyLevel.ONE);
>      }
> 
>      StorageService.instance.forceTableFlush(CassandraInitializer.KS,
> "TestCassandra");
> 
>      Thread.sleep(30*1000);
>   }
> 
> 
> ===exception when running test=========================================
> 
> 13:54:38 ERROR [PrecompactedRow.<init>]  Skipping row
> DecoratedKey(rowId, 726f774964) in
> ./target/test/cassandra/data/Test/TestCassandra-e-4-Data.db
> org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
> column name length 0
>    at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:68)
>    at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
>    at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
>    at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:137)
>    at
> org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78)
>    at
> org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
>    at
> org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
>    at
> org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42)
>    at
> org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
>    at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:137)
>    at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:132)
>    at
> org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
>    at
> org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
>    at
> org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:312)
>    at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
>    at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
>    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>    at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>    at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>    at java.lang.Thread.run(Thread.java:662)
> 
> ===yaml file============================================
> 
> # Cassandra YAML generated from previous config
> # Configuration wiki: http://wiki.apache.org/cassandra/StorageConfiguration
> authenticator: org.apache.cassandra.auth.AllowAllAuthenticator
> auto_bootstrap: false
> binary_memtable_throughput_in_mb: 256
> cluster_name: CluperUnitTest
> column_index_size_in_kb: 64
> commitlog_directory: ./target/test/cassandra/commitlog
> commitlog_rotation_threshold_in_mb: 128
> commitlog_sync: periodic
> commitlog_sync_period_in_ms: 10000
> compaction_thread_priority: 1
> concurrent_reads: 8
> concurrent_writes: 32
> data_file_directories:
> - ./target/test/cassandra/data
> disk_access_mode: auto
> dynamic_snitch: false
> dynamic_snitch_badness_threshold: 0.0
> dynamic_snitch_reset_interval_in_ms: 600000
> dynamic_snitch_update_interval_in_ms: 100
> endpoint_snitch: org.apache.cassandra.locator.RackInferringSnitch
> hinted_handoff_enabled: true
> in_memory_compaction_limit_in_mb: 256
> index_interval: 128
> saved_caches_directory: ./target/test/cassandra/saved_caches
> keyspaces:
> - column_families:
>  - column_metadata: []
>    column_type: Standard
>    compare_with: UTF8Type
>    gc_grace_seconds: 864000
>    key_cache_save_period_in_seconds: 3600
>    keys_cached: 200000.0
>    max_compaction_threshold: 32
>    memtable_flush_after_mins: 60
>    memtable_operations_in_millions: 0.3
>    memtable_throughput_in_mb: 64
>    min_compaction_threshold: 4
>    name: TestCassandra
>    read_repair_chance: 1.0
>    row_cache_save_period_in_seconds: 0
>    rows_cached: 0.0
>  name: Test
>  replica_placement_strategy: org.apache.cassandra.locator.SimpleStrategy
>  replication_factor: 1
> listen_address: localhost
> partitioner: org.apache.cassandra.dht.OrderPreservingPartitioner
> phi_convict_threshold: 8
> rpc_keepalive: true
> rpc_port: 9161
> rpc_timeout_in_ms: 10000
> seeds:
> - 127.0.0.1
> sliced_buffer_size_in_kb: 64
> snapshot_before_compaction: false
> storage_port: 7012
> thrift_framed_transport_size_in_mb: 15
> thrift_max_message_length_in_mb: 16
> 
> ===start up exception after unit test===================================
> 
> 14:16:14 ERROR [AbstractCassandraDaemon.activate]  Exception encountered
> during startup.
> org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
> column name length 0
> org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
> column name length 0
>    at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:68)
>    at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
>    at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:68)
>    at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
>    at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
>    at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:120)
>    at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
>    at
> org.apache.cassandra.db.RowMutationSerializer.defreezeTheMaps(RowMutation.java:385)
>    at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:120)
>    at
> org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:395)
>    at
> org.apache.cassandra.db.RowMutationSerializer.defreezeTheMaps(RowMutation.java:385)
>    at
> org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:353)
>    at
> org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:395)
>    at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:283)
>    at
> org.apache.cassandra.db.RowMutationSerializer.deserialize(RowMutation.java:353)
>    at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:194)
>    at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:283)
>    at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:143)
>    at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:194)
>    at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:55)
>    at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:143)
>    at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:217)
>    at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:55)
>    at
> senselogic.cluper.cassandra.CassandraInitializer$1.run(CassandraInitializer.java:67)
>    at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:217)
>    at java.lang.Thread.run(Thread.java:662)
>    at
> senselogic.cluper.cassandra.CassandraInitializer$1.run(CassandraInitializer.java:67)
> Exception encountered during startup.
>    at java.lang.Thread.run(Thread.java:662)'
> 
> ===ls of the commit log directory after the error=================
> 
> mikael@mikael2:~/projects/sitevision/trunk/framework/sitevision-framework-core/target/test-classes/target/test/cassandra/commitlog>
> ls -ltr
> total 187936
> -rw-r--r-- 1 mikael wheel 134217730 2011-02-01 14:13
> CommitLog-1296565996491.log
> -rw-r--r-- 1 mikael wheel        28 2011-02-01 14:13
> CommitLog-1296566017932.log.header
> -rw-r--r-- 1 mikael wheel        52 2011-02-01 14:13
> CommitLog-1296565996491.log.header
> -rw-r--r-- 1 mikael wheel  58211952 2011-02-01 14:13
> CommitLog-1296566017932.log
> -rw-r--r-- 1 mikael wheel         0 2011-02-01 14:16
> CommitLog-1296566174499.log
> -rw-r--r-- 1 mikael wheel        20 2011-02-01 14:16
> CommitLog-1296566174499.log.header
> 
> -- 
> Mikael Wikblom
> Software Architect
> SiteVision AB
> 019-217058
> mikael.wikblom@sitevision.se
> http://www.sitevision.se
> 

Mime
View raw message