incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charlie Mason <charlie....@gmail.com>
Subject Consolidating records and TTL
Date Tue, 03 Jun 2014 19:13:55 GMT
Hi All.

I have a system thats going to make possibly several concurrent changes to
a running total. I know I could use a counter for this. However I have
extra meta data I can store with the changes which would allow me to reply
the changes. If I use a counter and it looses some writes I can't recover
it as I will only have its current total not the extra meta data to know
where to replay from.

What I was planning to do was write each change of the value to a CQL table
with a Time UUID as a row level primary key as well as a partition key.
Then when I need to read the running total back I will do a query for all
the changes and add them up to get the total.

As there could be tens of thousands of these I want to have a period after
which these are consolidated. Most won't be any where near that but a few
will which I need to be able to support. So I was also going to have a
consolidated total table which holds the UUID of the values consolidated up
to. Since I can bound the query for the recent updates by the UUID I should
be able to avoid all the tombstones. So if the read encounters any changes
that can be consolidated it inserts a new consolidated value and deletes
the newly consolidated changes.

What I am slightly worried about is what happens if the consolidated value
insert fails but the deletes to the change records succeed. I would be left
with an inconsistent total indefinitely. I have come up with a couple of
ideas:


1, I could make it require all nodes to acknowledge it before deleting the
difference records.

2, May be I could have another period after its consolidated but before its
deleted?

3, Is there anyway I could use the TTL to allow to it to be deleted after a
period of time? Chances are another read would come in and fix the value.


Anyone got any other suggestions on how I could implement this?


Thanks,

Charlie M

Mime
View raw message