Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D1D67173FC for ; Fri, 10 Apr 2015 17:17:44 +0000 (UTC) Received: (qmail 98013 invoked by uid 500); 10 Apr 2015 17:17:41 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 97973 invoked by uid 500); 10 Apr 2015 17:17:41 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 97962 invoked by uid 99); 10 Apr 2015 17:17:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Apr 2015 17:17:41 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of agarwalpranaya@gmail.com designates 74.125.82.43 as permitted sender) Received: from [74.125.82.43] (HELO mail-wg0-f43.google.com) (74.125.82.43) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Apr 2015 17:17:17 +0000 Received: by wgin8 with SMTP id n8so24126958wgi.0 for ; Fri, 10 Apr 2015 10:16:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Pz7TV6bdXa68wIU8sVOaqn7NHzJi+ZqZODZkiqPFRVQ=; b=bbFjA3Ul7fdC4rAptH3eRfAK1ih6YXkMAOlLa+w3kBC2fhpw+att39S8YU/NeX1lNX cSWCAEaESh7eSta4ao+6lVOB0I5YA8CWxbbieguw3QMsDPP8pf65H/nnDhe/S8cGpl9Q 42r09simuPgFRPQj10pqRk3fUkxF+BIZZfRuulf2TsHb3zsg1KuqXhXec6E9sKaLJ3Td AvcsRO1lbTOgpO55qgKyhkKNKSY6hC2XQlhYEDtsEQBwgi29g4IEucPfuhrG8bDO0jXV Q+1r+e9nqYHD0xdmN9/CTHQ/qw4b4oRYsE0mecbX1WmPJpF65bOyQBEI1kxUgXLQ92W1 uHbA== MIME-Version: 1.0 X-Received: by 10.180.87.101 with SMTP id w5mr6616393wiz.18.1428686190720; Fri, 10 Apr 2015 10:16:30 -0700 (PDT) Received: by 10.180.96.42 with HTTP; Fri, 10 Apr 2015 10:16:30 -0700 (PDT) In-Reply-To: References: <1428630475.53523.YahooMailAndroidMobile@web192906.mail.sg3.yahoo.com> Date: Fri, 10 Apr 2015 10:16:30 -0700 Message-ID: Subject: Re: Cassandra Data Loss From: Pranay Agarwal To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=f46d044402e4e4bcac051361ecb1 X-Virus-Checked: Checked by ClamAV on apache.org --f46d044402e4e4bcac051361ecb1 Content-Type: text/plain; charset=UTF-8 Thanks Anuj and Jens. So, my initial assumption is correct that cassandra will *attempt* to replicate data irrespective of the CL value. However, if those asynchronous calls to replicate is failed, is there any retrial done by cassandra? Will a full cluster wider node repair will take care this and guarantee the *all* data is now replicated RF times? We really don't care about reading stale data that much, but we really want the data to be guaranteed to be replicated 3 times, so that we don't loose the data even if 2 nodes fail. We are doing this heavy write/read as initial import and ideally I would like keep CL 1 so that client is not blocked but at the same time we want cassandra to take care in background or asynchronously and ensure replication. On Fri, Apr 10, 2015 at 1:02 AM, Jens Rantil wrote: > Somewhat related: http://wiki.apache.org/cassandra/ReadRepair states > > Range scans are not per-key and do not do read repair. > > > Does "key" in "per-key" refer to "partition key" or "partition+clustering > key"? > > Cheers, > Jens > > On Fri, Apr 10, 2015 at 3:47 AM, Anuj Wadehra > wrote: > >> Read repair and repair run as part of maintenance will make it >> consistent. Read repair is usually done on only 10% of reads. You can tune >> tune read_repair_chance property of cf to adjust that. Till a row is >> repaired clients may return stale data if cl=1 is used for reads. I would >> suggest that u should minimize dropping of mutations by tuning if thats the >> case rather than fixing it. >> >> Thanks >> Anuj Wadehra >> >> Sent from Yahoo Mail on Android >> >> ------------------------------ >> *From*:"Kurtis vel" >> *Date*:Fri, 10 Apr, 2015 at 7:09 am >> *Subject*:Re: Cassandra Data Loss >> >> Hi Anuj, >> >> Assuming cl=1 and rf=3. >> >> Will the data ever be consistent if an asynchronous replication call >> fails? >> >> Is this where read repair comes in handy? >> >> thanks >> >> On Thu, Apr 9, 2015 at 6:24 PM, Anuj Wadehra >> wrote: >> >>> Cl=1 means that client will only block for one response. In case of >>> writes other 2 replicas will be updated asynchronously and eventually >>> updated. As you are running heavy load make sure that writes /mutations are >>> not getting dropped using nodetool tpstats on all nodes. Under heavy loads >>> Cassandra may drop writes and as these were asynchronous,client wont know >>> about that. >>> >>> if cl=1 for both reads and writes. Some reads may return stale data.If >>> you need absolute guarantee that reads always return up to date data go for >>> strong consistency r cf + w cf greater than rf. Eg read at quorum and write >>> at quorum. >>> >>> Thanks >>> Anuj Wadehra >>> >>> Sent from Yahoo Mail on Android >>> >>> ------------------------------ >>> *From*:"Pranay Agarwal" >>> *Date*:Fri, 10 Apr, 2015 at 6:40 am >>> *Subject*:Cassandra Data Loss >>> >>> Hi All. >>> >>> >>> I am using 20 nodes cassandra cluster with RF=3 and CL=1. We are doing >>> very write/read heavy operations (total 100k ops/sec). >>> >>> I have been assuming all along that all the data will be replicated in 3 >>> different place *irrespective of consistency level *as it's a very >>> application/driver level config. Is that correct or Cassandra guarantees 3 >>> replica only when I also have CL as 3 as well? >>> >>> >>> Thanks >>> -Pranay >>> >> >> > > > -- > Jens Rantil > Backend engineer > Tink AB > > Email: jens.rantil@tink.se > Phone: +46 708 84 18 32 > Web: www.tink.se > > Facebook Linkedin > > Twitter > --f46d044402e4e4bcac051361ecb1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Anuj and Jens.

So, my initial as= sumption is correct that cassandra will *attempt* to replicate data irrespe= ctive of the CL value. However, if those asynchronous calls to replicate is= failed, is there any retrial done by cassandra? Will a full cluster wider = node repair will take care this and guarantee the *all* data is now replica= ted RF times?

We really don't care about readi= ng stale data that much, but we really want the data to be guaranteed to be= replicated 3 times, so that we don't loose the data even if 2 nodes fa= il. We are doing this heavy write/read as initial import and ideally I woul= d like keep CL 1 so that client is not blocked but at the same time we want= cassandra to take care in background or asynchronously and ensure replicat= ion.

O= n Fri, Apr 10, 2015 at 1:02 AM, Jens Rantil <jens.rantil@tink.se&g= t; wrote:
Somewha= t related:=C2=A0http://wiki.apache.org/cassandra/ReadRepair states
<= br>
Range scans are not per-key and do not do read repair= .

Does "key" in "per-= key" refer to "partition key" or "partition+clustering = key"?

Cheers,
Jens
<= div class=3D"gmail_extra">

On Fri, Apr 10, 2015 at 3:47 AM, Anuj Wadehra <anujw_2003@yahoo.= co.in> wrote:
Re= ad repair and repair run as part of maintenance will make it consistent. Re= ad repair is usually done on only 10% of reads. You can tune tune read_repa= ir_chance property of cf to adjust that. Till a row is repaired clients may= return stale data if cl=3D1 is used for reads. I would suggest that u shou= ld minimize dropping of mutations by tuning if thats the case rather than f= ixing it.

Thanks
Anuj Wadehra

Sent fr= om Yahoo Mail on Android

From:"Kurtis vel&quo= t; <kurtisv= elarde@gmail.com>
Date:Fri, 10 Apr, 2015 at 7:09 am
Subject:Re: Cassandra Data Loss

Hi Anuj,

Assuming cl= =3D1 and rf=3D3.

Will the data ever be consiste= nt if an asynchronous replication call fails?

Is this where read repair comes in handy?

thanks

On Thu, Ap= r 9, 2015 at 6:24 PM, Anuj Wadehra <anujw_2003@yahoo.co.in> wrote:
= Cl=3D1 means that client will only block for one response. In case of write= s other 2 replicas will be updated asynchronously and eventually updated. A= s you are running heavy load make sure that writes /mutations are not getti= ng dropped using nodetool tpstats on all nodes. Under heavy loads Cassandra= may drop writes and as these were asynchronous,client wont know about that= .=C2=A0

if cl=3D1 for both reads and writ= es. Some reads may return stale data.If you need absolute guarantee that re= ads always return up to date data go for strong consistency r cf + w cf gre= ater than rf. Eg read at quorum and write at quorum.

Thanks



--
Jens Rantil
Backend eng= ineer
Tink AB

Phone: +46 708 84 18 32
Web:=C2= =A0www.tink.se
--f46d044402e4e4bcac051361ecb1--