incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Morton <dan...@djmorton.com>
Subject Re: Bulk loading into CQL3 Composite Columns
Date Thu, 30 May 2013 20:34:39 GMT
Hi Edward... Thanks for the pointer.  I will use that going forward.

Daniel Morton


On Thu, May 30, 2013 at 4:09 PM, Edward Capriolo <edlinuxguru@gmail.com>wrote:

> You should probably be using system.nanoTime() not
> system.currentTimeInMillis(). The user is free to set the timestamp to
> whatever they like but nano-time is the standard (it is what the cli uses,
> and what cql will use)
>
>
> On Thu, May 30, 2013 at 3:33 PM, Keith Wright <kwright@nanigans.com>wrote:
>
>> Sorry, typo in code sample, should be:
>>
>> ssTableWriter.newRow(StringSerializer.get().toByteBuffer("20101201"));
>> Composite columnComposite = new Composite();
>> columnComposite.setComponent(0,5,IntegerSerializer.get());
>> columnComposite.setComponent(1,10,IntegerSerializer.get());
>>
>> ssTableWriter.addColumn( CompositeSerializer.get().toByteBuffer(columnComposite),
null, System.currentTimeMillis() );
>>
>> From: Keith Wright <kwright@nanigans.com>
>> Date: Thursday, May 30, 2013 3:32 PM
>> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
>> Subject: Re: Bulk loading into CQL3 Composite Columns
>>
>> You do not want to repeat the first item of your primary key again.  If
>> you recall, in CQL3 a primary key as defined below indicates that the row
>> key is the first item (key) and then the column names are composites of
>> val1,val2.  Although I don't see why you need val2 as part of the primary
>> key in this case.  In any event, you would do something like this (although
>> I've never tested passing a null value):
>>
>> ssTableWriter.newRow(StringSerializer.get().toByteBuffer("20101201"));
>> Composite columnComposite = new Composite();
>> columnComposite(0,5,IntegerSerializer.get());
>> columnComposite(0,10,IntegerSerializer.get());
>> ssTableWriter.addColumn(
>> CompositeSerializer.get().toByteBuffer(columnComposite),
>> null,
>> System.currentTimeMillis()
>> );
>>
>> From: Daniel Morton <daniel@djmorton.com>
>> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
>> Date: Thursday, May 30, 2013 1:06 PM
>> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
>> Subject: Bulk loading into CQL3 Composite Columns
>>
>> Hi All.  I am trying to bulk load some data into a CQL3 table using the
>> sstableloader utility and I am having some difficulty figuring out how to
>> use the SSTableSimpleUnsortedWriter with composite columns.
>>
>> I have created this simple contrived table for testing:
>>
>> create table test (key varchar, val1 int, val2 int, primary key (key,
>> val1, val2));
>>
>> Loosely following the bulk loading example in the docs, I have
>> constructed the following method to create my temporary SSTables.
>>
>> public static void main(String[] args) throws Exception {
>>    final List<AbstractType<?>> compositeTypes = new ArrayList<>();
>>    compositeTypes.add(UTF8Type.instance);
>>    compositeTypes.add(IntegerType.instance);
>>    compositeTypes.add(IntegerType.instance);
>>    final CompositeType compType =
>>       CompositeType.getInstance(compositeTypes);
>>    SSTableSimpleUnsortedWriter ssTableWriter =
>>       new SSTableSimpleUnsortedWriter(
>>          new File("/tmp/cassandra_bulk/bigdata/test"),
>>          new Murmur3Partitioner() ,
>>          "bigdata",
>>          "test",
>>          compType,
>>          null,
>>          128);
>>
>>    final Builder builder =
>>       new CompositeType.Builder(compType);
>>
>>    builder.add(bytes("20101201"));
>>    builder.add(bytes(5));
>>    builder.add(bytes(10));
>>
>>    ssTableWriter.newRow(bytes("20101201"));
>>    ssTableWriter.addColumn(
>>          builder.build(),
>>          ByteBuffer.allocate(0),
>>          System.currentTimeMillis()
>>    );
>>
>>    ssTableWriter.close();
>> }
>>
>> When I execute this method and load the data using sstableloader, if I do
>> a 'SELECT * FROM test' in cqlsh, I get the results:
>>
>> key      | val1       | val2
>> ----------------------------
>> 20101201 | '20101201' | 5
>>
>> And the error:  Failed to decode value '20101201' (for column 'val1') as
>> int.
>>
>> The error I get makes sense, as apparently it tried to place the key
>> value into the val1 column.  From this error, I then assumed that the key
>> value should not be part of the composite type when the row is added, so I
>> removed the UTF8Type from the composite type, and only added the two
>> integer values through the builder, but when I repeat the select with that
>> data loaded, Cassandra throws an ArrayIndexOutOfBoundsException in the
>> ColumnGroupMap class.
>>
>> Can anyone offer any advice on the correct way to insert data via the
>> bulk loading process into CQL3 tables with composite columns?  Does the
>> fact that I am not inserting a value for the columns make a difference?
>>  For my particular use case, all I care about is the values in the column
>> names themselves (and the associated sorting that goes with them).
>>
>> Any info or help anyone could provide would be very much appreciated.
>>
>> Regards,
>>
>> Daniel Morton
>>
>
>

Mime
View raw message