cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jimmy Lin <>
Subject Re: read repair across DC and latency
Date Wed, 19 Nov 2014 22:51:45 GMT
thanks for the detail explanation.
Still have few questions in my mind....

When you said send "read digest request" to the rest of the replica, do you
mean all replica(s) in current and other DC? or just the one last replica
in my current DC and one of the co-ordinate node in other DC?

(our read and write is all "local_quorum" of replication factor of 3,

Sending "read digest request" to other DC, happen sequently correct? If
network latency between DC is bad during time, will that affect overall
read latency?

We observe that one of our cql query perform okay during normal load, but
degrade greatly when we have batch of  same cql(looking for the exact
columns and key) sending to server in short period of time(say 100 of them
within a sec).
Our other table or keyspace don't see any latency drop during the time, so
i am not sure we are hitting the capacity yet. So we suspect read_repair
chance may have something to do wit it.
Anything we can look into and see what may cause the latency spike when we
have large number of same cql hitting the server?


On Wed, Nov 19, 2014 at 7:49 AM, Tyler Hobbs <> wrote:

> On Sun, Nov 16, 2014 at 5:13 PM, Jimmy Lin <> wrote:
>> I have read  that read repair suppose to be running as background, but
>> does the co-ordinator node need to wait for the response(along with other
>> normal read tasks) before return the entire result back to the caller?
> For the 10% of requests where read repair is triggered, the coordinator
> will send a request to every replica.  (A data request to two replicas,
> digest requests to the rest.)  Once enough replicas have replied to satisfy
> the consistency level, the result will be returned to the client; if
> there's a mismatch in the responses from the replicas, a blocking repair
> will be performed before responding to the client.  Later, in the
> background, the coordinator will check the remaining responses from
> replicas to see if they match up.  If any of them do not, they will be
> repaired in the background.
>> #
>> how a high rate of read repair impact performance? I read something that
>> it will impact through put but not latency, how so?
> That's correct, it should impact throughput but not necessarily latency.
> Throughput is lower because more replicas have to do work, but latency is
> unaffected (unless you're hitting capacity) because blocking repair only
> happens under the same conditions that it normally does.
>> #
>> is it safe to even just  make read_repair_chance = 0?
>> (since we are mostly talking to one DC, the other DC most of the time
>> serve as backup/emergency )
> Sure, it's safe enough.  People use read repair for different reasons.
> Some would say that RR keeps their other datacenter's caches warm. Others
> rely on it in place of normal repairs (which is not particularly safe, but
> if your consistency requirements allow for it, it's fine).  If you're
> running regular repairs anyway, it's safe to turn off read repair.
> --
> Tyler Hobbs
> DataStax <>

View raw message