cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominic Williams <dwilli...@system7.co.uk>
Subject Re: No Transactions: An Example
Date Wed, 22 Jun 2011 17:04:55 GMT
Hi Trevor,

I hope to post on my practical experiences in this area soon - we rely
heavily on complex serialized operations in FightMyMonster.com. Probably the
most simple serialized operation we do is updating nugget balances when, for
example, there has been a trade of monsters.

Currently we use ZooKeeper/Cages (github.com/s7) to serialize our
distributed ops.

We don't implement transactions with rollback/commit. Rather, we lock some
paths, for example /Users/bank/dominic and /Users/bank/ben, and then write
with QUORUM through our Java client library Pelops. This will make several
efforts to retry the operation if it fails at first, and in our line of
business the fact that redundancy in the cluster means it will nearly always
complete eventually is enough.

Of course, in a real world money scenario that is not enough and data
inconsistency caused by, say, a sudden power outage during the retry phase
is not acceptable. To handle this case I would like to extend Cages at some
point so that commit/rollback transactions that would be stored inside
ZooKeeper are associated with the distributed locks (which are stored
persistently and survive power loss for example). There is an old blog post
here which talks about it
http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/although
this needs updating.

One interesting point not discussed which I have also not heard mentioned
elsewhere is that in order for serialization to work every time, before you
release a lock after performing an update you must wait for a brief period
>= max variance between the clocks on the application nodes updating the
database e.g. 1-2ms.

This is because Cassandra uses the timestamps of columns that have been
written during reconciliation to determine which should be persisted when
they conflict.

As far as scaling goes, ZooKeeper can be scaled by having several clusters
and hashing lock paths to them. Alternatively, Lamport's bakery algorithm
could be investigated as this shows you can have locking without a central
coordinator service.

Best, Dominic

On 22 June 2011 15:18, Trevor Smith <trevor@knewton.com> wrote:

> Hello,
>
> I was wondering if anyone had architecture thoughts of creating a simple
> bank account program that does not use transactions. I think creating an
> example project like this would be a good thing to have for a lot of the
> discussions that pop up about transactions and Cassandra (and
> non-transactional datastores in general).
>
> Consider the simple system that has accounts, and users can transfer money
> between the accounts.
>
> There are these interesting papers as background (links below).
>
>  Thank you.
>
> Trevor Smith
>
> http://www.ics.uci.edu/~cs223/papers/cidr07p15.pdf
>
>
> http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-components-postattachments/00-09-20-52-14/BuildingOnQuicksand_2D00_V3_2D00_081212h_2D00_pdf.pdf
>
> http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf
>

Mime
View raw message