incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From T Akhayo <t.akh...@gmail.com>
Subject Importing sstable with Composite key? (without is working)
Date Thu, 04 Oct 2012 18:15:13 GMT
Good evening,

Today i managed to get a small cluster running of 2 computers. I also
managed to get my data model working and are able to import sstables
created with SSTableSimpleUnsortedWriter with sstableloader.

The only problem is when i try to use the composite key in my datamodel,
after i import my sstables and issue a simple select the cassandra crashes:
===
ava.lang.IllegalArgumentException
        at java.nio.Buffer.limit(Unknown Source)
        at
org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:51)
        at
org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:60)
        at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:76)
        at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31)
        at java.util.TreeMap.put(Unknown Source)
        at
org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:95)
        at
org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
...
        at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:108)
        at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:121)
        at
org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1237)
        at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3542)
        at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3530)
        at
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
        at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
        at java.lang.Thread.run(Unknown Source)
===

Now i can get everything running again by removing the data directories on
both nodes.

I suspect cassandra crashes because the sstable that is being imported has
a different schema when it comes to composite key (without composite key
import works fine).

My schema with composite key is:
===
create table bars2(
id uuid,
timeframe int,
datum timestamp,
open double,
high double,
low double,
close double,
bartype int,
PRIMARY KEY (timeframe, datum)
);
===
create column family bars2
  with column_type = 'Standard'
  and comparator =
'CompositeType(org.apache.cassandra.db.marshal.DateType,org.apache.cassandra.db.marshal.UTF8Type)'
  and default_validation_class = 'UTF8Type'
  and key_validation_class = 'Int32Type'
  and read_repair_chance = 0.1
  and dclocal_read_repair_chance = 0.0
  and gc_grace = 864000
  and min_compaction_threshold = 4
  and max_compaction_threshold = 32
  and replicate_on_write = true
  and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
  and caching = 'KEYS_ONLY'
  and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
===

My code to create the sstable is (only the interested parts):
===
sstWriter = new SSTableSimpleUnsortedWriter(new
File("c:\\cassandra\\newtables\\"), new RandomPartitioner(), "readtick",
                    "bars2", UTF8Type.instance, null, 64);
....

CompositeType.Builder cb=new
CompositeType.Builder(CompositeType.getInstance(compositeList));
            cb.add( bytes(curMinuteBar.getDatum().getTime()));
            cb.add(bytes(1));
            sstWriter.newRow(cb.build());

            (... add columns...)
===

I highly suspect that the problem can be at 2 locations:
- In the SSTableSimpleUnsortedWriter i use a UTF8Type.instance as
comparator, i'm not sure if that is right with a composite key?
- When calling "sstWriter.newRow" i use "CompositeType.Builder" to build
the composite key, i'm not sure if i'm doing this the right way? (i did try
different combinations)

Does somebody know how i can continue on my journey?

Mime
View raw message