cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Desimpel, Ignace" <>
Subject RE: ordered partitioner
Date Tue, 23 Apr 2013 08:28:59 GMT
I got into problems on starting a new database. That starts up ok. Then I add a keyspace and
it goes wrong. The error came from DefsTable.mergeSchema (working on version 1.2.1). It starts
by fetching the current keyspace and stores it in a map of DecoratedKey elements. Then it
applies the mutation and fetches all the keyspaces again. But now the fetch gets a map of
the new OwnDecoratedKey elements. So the difference of these elements contains all the 'old'
DecoratedKey and the 'new' OwnDecoratedKey elements, each kind of DecoratedKey contains all
the old existing keyspaces. So the the mergeKeyspaces routine thinks that it has to recreate
the already existing keyspaces like system_auth , etc...

Currently in my schema, I don't use indexes, so don't know the effect. Also, the change (local
of course) I did, was done without really knowing what to do (kind of blind replacement) ...
But the one reference in SecondaryIndex was left as it is, since it was using a LocalToken

What I changed to make it work is :
1) Function Memtable:: resolve(DecoratedKey key, ColumnFamily cf, SecondaryIndexManager.Updater
  With line "previous = columnFamilies.putIfAbsent(new DecoratedKey(key.token, allocator.clone(key.key)),
  Changed to "previous = columnFamilies.putIfAbsent(StorageService.getPartitioner().decorateKey(allocator.clone(key.key)),
2) Function SSTable:: getMinimalKey(DecoratedKey key)
  With line " new DecoratedKey(key.token, HeapAllocator.instance.clone(key.key))"
  Changed to " StorageService.getPartitioner().decorateKey(HeapAllocator.instance.clone(key.key))
3) ColumnFamilyStore:: getSSTablesForKey(String key)
  With line "DecoratedKey dk = new DecoratedKey(partitioner.getToken(ByteBuffer.wrap(key.getBytes())),
  Changed to DecoratedKey dk = partitioner.decorateKey(ByteBuffer.wrap(key.getBytes()));



-----Original Message-----
From: Jonathan Ellis [] 
Sent: maandag 22 april 2013 20:12
To: user
Subject: Re: ordered partitioner

Not in general, no.  There are places, like indexing, that need to use a local partitioner
rather than the global one.

Which uses of the DK constructor looked erroneous to you?

On Mon, Apr 22, 2013 at 10:54 AM, Desimpel, Ignace <> wrote:
> Hi,
> I was trying to implement my own ordered partitioner and got into problems.
> The current DecoratedKey is using a ByteBufferUtil.compareUnsigned for 
> comparing the key. I was thinking of having a signed comparison, so I 
> thought of making my own DecoratedKey, Token and Partitioner. That way 
> I would have complete control...
> So  made a partitioner whith a function decorateKey(...) returning 
> MyDecoratedKey in stead of DecoratedKey
> But when making my own MyDecoratedKey, the database get into trouble 
> when adding a key space due to the fact that some code in Cassandra is 
> using the 'new DecoratedKey(...)' statement and is not using the 
> partitioner function decorateKey(...).
> Would it be logical to always call the partitioner function 
> decorateKey such that the creation of an own partitioner and key decoration is possible?
> Ignace Desimpel

Jonathan Ellis
Project Chair, Apache Cassandra

View raw message