cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Hendry <dan.hendry.j...@gmail.com>
Subject Re: Confused about consistency
Date Sat, 04 Dec 2010 06:01:19 GMT
Doesn't consistency level ALL=QUORUM at RF=2 ?

I have not had a chance to test your fix but I don't THINK this is the
issue. If it is the issue, how do consistency levels ALL and QUORUM differ
at this replication factor?

On Sat, Dec 4, 2010 at 12:03 AM, Jonathan Ellis <jbellis@gmail.com> wrote:

> I think you are running into
> https://issues.apache.org/jira/browse/CASSANDRA-1316, where when an
> inconsistency on QUORUM/ALL is discovered it always peformed the
> repair at QUORUM instead of the original CL.  Thus, reading at ALL you
> would see the correct answer on the 2nd read but you weren't
> guaranteed to see it on the first.
>
> This was fixed in 0.6.4 but apparently I botched the merge to the 0.7
> branch.  I corrected that just now, so when you update, you should be
> good to go.
>
> On Fri, Dec 3, 2010 at 9:19 PM, Dan Hendry <dan.hendry.junk@gmail.com>
> wrote:
> > I am seeing fairly strange, behavior in my Cassandra cluster.
> > Setup
> >  - 3 nodes (lets call them nodes 1 2 and 3)
> >  - RF=2
> >  - A set of servers (producers) which which write data to the cluster at
> > consistency level ONE
> >  - A set of servers (consumers/processors) which read data from the
> cluster
> > at consistency level ALL
> >  - Cassandra 0.7 (recent out of the svn branch, post beta 3)
> >  - Clients use the pelops library
> > Situation:
> >  - Everything is humming along nicely
> >  - A Cassandra node (say 3) goes down (even with 24 GB of ram, OOM errors
> > are the bain of my existence)
> >  - Producers continue to happily write to the cluster but consumers start
> > complaining by throwing TimeOutExceptions and UnavailableExceptions.
> >  - I stagger out of bed in the middle of the night and restart Cassandra
> on
> > node 3.
> >  - The consumers stop complaining and get back to business but generate
> > garbage data for the period node 3 was down. Its almost like half the
> data
> > is missing half the time. (Again, I am reading at consistency level ALL).
> >  - I force the consumers to reprocess data for the period node 3 was
> down.
> > They generate accurate output which is different from the first time
> round.
> > To be explicit, what seems to be happening is first read at consistency
> ALL
> > gives "A,C,E" (for example) and the second read at consistency level ALL
> > gives "A,B,C,D,E". Is this a Cassandra bug? Is my knowledge of
> consistency
> > levels flawed? My understanding is that you could achieve strongly
> > consistent behavior by writing at ONE and reading at ALL.
> > After this experience, my theory (uneducated, untested, and
> > under-researched) is that "strong consistency" applies only to column
> > values, not the set of columns (or super-columns in this case) which make
> up
> > a row. Any thoughts?
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>

Mime
View raw message