On Wed, Aug 3, 2011 at 4:00 PM, Philippe <watcherfr@gmail.com> wrote:
I have a 3-node, RF=3, cluster configured to write at CL.ALL and read at CL.ONE. When I take one of the nodes down, writes fail which is what I expect.
When I run a repair, I see data being streamed from those column families... that I didn't expect. How can the nodes diverge ? Does this mean that reading at CL.ONE may return inconsistent data ?

we abort the mutation before hand when there are enough replicas alive. If a mutation went through and in the middle of it a replica goes down, in that case you can write to some nodes and the request will Timeout.
In that case the CL.ONE may return inconsistence data. 

Question 2 : I've doing this rather than CL.QUORUM because I've been expecting CL.ONE to return data faster than CL.QUORUM. Is that a good assumption ? Yes, it's ok for writes to be down for a while.

When you hit a node that own the piece of data, CL.ONE will be faster as you don't have to wait for a read across the network to reach another node.
For CL.QUORUM we fire reads in parallel to all the replicas and wait until completing quorum. If I'm not wrong, in some cases the difference may be negligible for CL.ONE and CL.QUORUM when you hit a coordinator that doesn't own the data since you are going  over the network anyway (assuming all nodes take the same time to reply)