incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Davis <jerdavis.cassan...@gmail.com>
Subject Creating a Total Ordered Queue in Cassandra
Date Fri, 02 Apr 2010 01:32:39 GMT
I'm in the process of implementing a Totally Ordered Queue in Cassandra, and
wanted to bounce my ideas off the list and also see if there are any other
suggestions.

I've come up with an external source of ID's that are always increasing (but
not monotonic), and I've also used external synchronization to ensure only
one writer to a given queue. And I handle de-duping in the app.


My current solution is : (simplified)

Use the "QueueId", to Key into a row of a CF.
Then, every column in that CF corresponds to a new entry in the Queue, with
a custom Comparator to sort the columns by my external ID that is always
increasing.

Technically I never delete data from the Queue, and I just page through it
from a given ID using a SliceRange, etc.

Obviously the problem being that the row needs to get compacted. so then I
started bucketizing with multiple rows for a given queue (for example one
per day (again I'm simplifying))...(so the Key is now "QueueId+Day"...)

Does this seem reasonable? It's solvable, but is starting to seem
complicated to implement... It would be very easy if I didn't have to have
multiple buckets..



My other thought is to store one entry per row, and perform get_range_slices
and specify a KeyRange, with the OrderPreservingPartitioner.
But it isn't exactly clear to me what the Order of the keys are in this
system, so I don't know how to construct my key and queries appropriately...
Is this Lexical String Order? Or?

So for example.. Assuming my QueueId's are longs, and my ID's are also
longs.. My key would be (in Java):

long queueId;
long msgId;

key = "" + queueId + ":" + msgId;

And if I wanted to do a query my key range might be from
start = "" + queueId + ":0"
end = "" + queueId + ":" + Long.MAX_VALUE;

(Will I have to left pad the msgIds with 0's)?

And is this going to be efficient if my msgId isn't monotonically
increasing?

Thanks,
-JD

Mime
View raw message