Is there a programmatic way to scrub and compact new sstable on an active node ?
Only to scrub all files using nodetool scrub. 

Cheers

-----------------
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton

On 1/02/2013, at 3:52 AM, POUGET Laurent <laurent.pouget@carboatmedia.fr> wrote:

Ok, in order to solve : java.lang.AssertionError: Added column does not sort as the last column
 
I just have to make a scrub with nodetools.
 
Is there a programmatic way to scrub and compact new sstable on an active node ?
 
Thx,
 
Laurent Pouget
 
 
De : POUGET Laurent [mailto:laurent.pouget@carboatmedia.fr] 
Envoyé : jeudi 31 janvier 2013 11:31
À : user@cassandra.apache.org
Objet : RE: SStable Writer and composite key
 
Thanks,
 
Import is ok now but :
·         If I query my table with 1.2.0, I have an exception :
 
java.lang.AssertionError: Added column does not sort as the last column
       at org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:131)
       at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
       at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:104)
       at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:171)
       at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:136)
       at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:84)
       at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:286)
       at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:61)
       at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1362)
       at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1222)
       at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1134)
       at org.apache.cassandra.db.Table.getRow(Table.java:348)
       at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70)
       at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1048)
       at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1506)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
       at java.lang.Thread.run(Unknown Source)
 
With 1.2.1 data doesn’t load :
 
ERROR  Corrupt sstable <path>/<keyspace_name>/<cf_name>/<cf_name>_logs-ib-42=[TOC.txt, Summary.db, Index
.db, Data.db, Filter.db, Digest.sha1, Statistics.db]; skipped
java.io.EOFException
        at java.io.DataInputStream.readFully(Unknown Source)
        at java.io.DataInputStream.readUTF(Unknown Source)
        at java.io.DataInputStream.readUTF(Unknown Source)
        at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:27
8)
        at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:25
1)
        at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:166)
        at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:149)
        at org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:238)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
 
 
Now with your advice, my java code looks like :
 
 
simpleUnsortedWriter.newRow(bytes(id));
            
ByteBuffer date = bytes(_date);
ByteBuffer request = bytes(_request);
 
long timestamp = System.currentTimeMillis() * 1000;
            
addColumn(date, request, "", "", timestamp);
addColumn(date, request, "data1", data1, timestamp);
addColumn(date, request, "data2", data2, timestamp);
 
 
public void addColumn(ByteBuffer date,
                    ByteBuffer request, String field, String value, long timestamp)
                    throws IOException {
       builder = compositeType.builder();
            
       builder.add(date);
       builder.add(request);
       builder.add(bytes(field));
            
       simpleUnsortedWriter.addColumn(builder.build(),
                    bytes(value), timestamp);
}
 
 
 
 
 
De : aaron morton [mailto:aaron@thelastpickle.com] 
Envoyé : jeudi 31 janvier 2013 00:03
À : user@cassandra.apache.org
Objet : Re: SStable Writer and composite key
 
This is what a row of your table will look like internally…
 
-------------------
RowKey: id-value
=> (column=date-value:request-value:, value=, timestamp=1359586739456000)
=> (column=date-value:request-value:data1, value=64617461312d76616c7565, timestamp=1359586739456000)
=> (column=date-value:request-value:data2, value=64617461322d76616c7565, timestamp=1359586739456000)
 
where id-value is the value of id column, and date-value is the….
 
So you need to construct triples for each column value. 
The first col name is the (date-value, request-value, empty) 
The second col name is (date-value, request-value, "data1") and has the value of data1. 
 
Hope that helps. 
 
-----------------
Aaron Morton
Freelance Cassandra Developer
New Zealand
 
@aaronmorton
 
On 30/01/2013, at 3:11 AM, POUGET Laurent <laurent.pouget@carboatmedia.fr> wrote:

 

Hi,
 
 
I have some trouble to request my data. I use SSTableSimpleUnsortedWriter to write SSTable. Writing and Importing works fine.
I think, I’m misusing CompositeType.Builder with SSTableSimpleUnsortedWriter.
Do you have any idea ?
 
Thanks
 
Here is my case :
 
/**
* CREATE STATEMENT
*/
 
CREATE TABLE raw_data (
  id text,
  date text,
  request text,
  data1 text,
  data2 text,
  PRIMARY KEY (id, date, request)
) WITH
  bloom_filter_fp_chance=0.010000 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.000000 AND
  gc_grace_seconds=864000 AND
  read_repair_chance=0.100000 AND
  replicate_on_write='true' AND
  compaction={'class': 'SizeTieredCompactionStrategy'} AND
  compression={'sstable_compression': 'SnappyCompressor'};
 
/**
* JAVA CODE
*/
 
List<AbstractType<?>> compositeList = new ArrayList<AbstractType<?>>();
 
compositeList.add(UTF8Type.instance);
compositeList.add(UTF8Type.instance);
 
IPartitioner<?> partitioner = StorageService.getPartitioner();
dir = Directories.create(keyspace.getKeyspaceName(), columnFamily.getName()).getDirectoryForNewSSTables(0);
 
simpleUnsortedWriter = new SSTableSimpleUnsortedWriter(dir,
                                                               partitioner, keyspace.getKeyspaceName(),
                                                               columnFamily.getName(), UTF8Type.instance, null,
                                                               32);
 
CompositeType.Builder builderRequestDate = new CompositeType.Builder( CompositeType.getInstance                       (compositeList) );
CompositeType.Builder builderUrl = new CompositeType.Builder( CompositeType.getInstance(compositeList) );
 
simpleUnsortedWriter.newRow(bytes(id));
                              
builderRequestDate.add(bytes("date"));
builderRequestDate.add(bytes("request"));
 
long timestamp = System.currentTimeMillis() * 1000;
 
simpleUnsortedWriter.addColumn(builderRequestDate.build(),
                               bytes(date), timestamp);
simpleUnsortedWriter.addColumn(builderUrl.build(),
                               bytes(request), timestamp);
 
simpleUnsortedWriter.addColumn(bytes("data1"),
                               bytes(data1), timestamp);
simpleUnsortedWriter.addColumn(bytes("data2"),
                               bytes(data2), timestamp);
 
simpleUnsortedWriter.close();
 
 
________________________________
 
Laurent Pouget
Ingénieur étude et développement
Tel : 01.84.95.11.20
 
Car & Boat Media
22 Rue Joubert  75009 Paris