incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Importing sstable with Composite key? (without is working)
Date Mon, 08 Oct 2012 19:28:38 GMT
Not sure why you have two different definitions for the bars2 CF. 

You will need to create SSTable's that match the schema cassandra has. 

Cheers
 
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 5/10/2012, at 7:15 AM, T Akhayo <t.akhayo@gmail.com> wrote:

> Good evening,
> 
> Today i managed to get a small cluster running of 2 computers. I also managed to get
my data model working and are able to import sstables created with SSTableSimpleUnsortedWriter
with sstableloader.
> 
> The only problem is when i try to use the composite key in my datamodel, after i import
my sstables and issue a simple select the cassandra crashes:
> ===
> ava.lang.IllegalArgumentException
>         at java.nio.Buffer.limit(Unknown Source)
>         at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:51)
>         at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:60)
>         at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:76)
>         at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31)
>         at java.util.TreeMap.put(Unknown Source)
>         at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:95)
>         at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
> ...
>         at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:108)
>         at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:121)
>         at org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1237)
>         at org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3542)
>         at org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3530)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
>         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
> ===
> 
> Now i can get everything running again by removing the data directories on both nodes.
> 
> I suspect cassandra crashes because the sstable that is being imported has a different
schema when it comes to composite key (without composite key import works fine).
> 
> My schema with composite key is:
> ===
> create table bars2(
> id uuid,
> timeframe int,
> datum timestamp,
> open double,
> high double,
> low double,
> close double,
> bartype int,
> PRIMARY KEY (timeframe, datum)
> );
> ===
> create column family bars2
>   with column_type = 'Standard'
>   and comparator = 'CompositeType(org.apache.cassandra.db.marshal.DateType,org.apache.cassandra.db.marshal.UTF8Type)'
>   and default_validation_class = 'UTF8Type'
>   and key_validation_class = 'Int32Type'
>   and read_repair_chance = 0.1
>   and dclocal_read_repair_chance = 0.0
>   and gc_grace = 864000
>   and min_compaction_threshold = 4
>   and max_compaction_threshold = 32
>   and replicate_on_write = true
>   and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
>   and caching = 'KEYS_ONLY'
>   and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'};
> ===
> 
> My code to create the sstable is (only the interested parts):
> ===
> sstWriter = new SSTableSimpleUnsortedWriter(new File("c:\\cassandra\\newtables\\"), new
RandomPartitioner(), "readtick",
>                     "bars2", UTF8Type.instance, null, 64);
> ....
> 
> CompositeType.Builder cb=new CompositeType.Builder(CompositeType.getInstance(compositeList));
>             cb.add( bytes(curMinuteBar.getDatum().getTime()));
>             cb.add(bytes(1));
>             sstWriter.newRow(cb.build());
>             
>             (... add columns...)
> ===
> 
> I highly suspect that the problem can be at 2 locations:
> - In the SSTableSimpleUnsortedWriter i use a UTF8Type.instance as comparator, i'm not
sure if that is right with a composite key?
> - When calling "sstWriter.newRow" i use "CompositeType.Builder" to build the composite
key, i'm not sure if i'm doing this the right way? (i did try different combinations)
> 
> Does somebody know how i can continue on my journey?
> 


Mime
View raw message