Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 77C91200BE1 for ; Mon, 19 Dec 2016 15:07:19 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 76490160B21; Mon, 19 Dec 2016 14:07:19 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 4C30B160B18 for ; Mon, 19 Dec 2016 15:07:18 +0100 (CET) Received: (qmail 13886 invoked by uid 500); 19 Dec 2016 14:07:16 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 13876 invoked by uid 99); 19 Dec 2016 14:07:16 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Dec 2016 14:07:16 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 58860C03A0 for ; Mon, 19 Dec 2016 14:07:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.189 X-Spam-Level: * X-Spam-Status: No, score=1.189 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=liveperson.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id slTTKxfqopJs for ; Mon, 19 Dec 2016 14:07:12 +0000 (UTC) Received: from mail-qk0-f173.google.com (mail-qk0-f173.google.com [209.85.220.173]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 175A75F4A9 for ; Mon, 19 Dec 2016 14:07:12 +0000 (UTC) Received: by mail-qk0-f173.google.com with SMTP id t184so15034246qkd.0 for ; Mon, 19 Dec 2016 06:07:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=liveperson.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Dh2qaLIoudHFIUlFiciJiEBlPWWdtDVcXnGhLPTQQf4=; b=gMPA96nvhBhZjAVKBoeMbRMRaLPFwa03xrkhxiGYYZNpbezUyp2a5AaKB+L08ZmdQo sg8Y3cbByKkVZurHJwN0HtJsn2bvlJlm4kid8JmsOpk1UFIoB71vICTey7faO4vLbbKq MV8ABPzeeCMtUx0j/RPo9NiahXel1kc/Zagtg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Dh2qaLIoudHFIUlFiciJiEBlPWWdtDVcXnGhLPTQQf4=; b=NXzuzlhE4NQwxN9FYGbVldzlgoQfNeZOEL/3GLdu7a+4++Lmf/yekrnl98rDndDWR4 1HUEU20VvGjuzUzidu47XSfWhBah58+u7ESALJoIiUWV0e1b5WHoT61TtT8yqqokyGb2 UnwnoAanBpjETbPJ2dnAvzQb5D7NPKAMh7DDv4SAlqSUaW2fZwoRrFbRjL983V8Nh1cc EzHL7lpEijUG1klt+EwITyUh4SjzTJLKd3myH9dEU8o6Dm7bEkUbs64h53hUZOz87YEh LRx5+FvCn9BGrkJlRuomAk+gBjLFIPFdjsKYTYlWIjwj3js2hc9Z+AnHzzcHHIg7iIF2 0jEg== X-Gm-Message-State: AIkVDXIj2+1brbdWK3B8YXkl7m+xGxCbEGzoILTfxlWey/vEGbDotySsXa9q950pB0uhvqbk52s7rMocOcpICiIxPDqBaUbNEff4ESSqDYwfpm2TWIgt9XnLpCPbvRQs1wrD82QIDvw+wJPPoato4ds= X-Received: by 10.55.141.199 with SMTP id p190mr364244qkd.232.1482156424292; Mon, 19 Dec 2016 06:07:04 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.20.35 with HTTP; Mon, 19 Dec 2016 06:07:03 -0800 (PST) In-Reply-To: References: From: Shalom Sagges Date: Mon, 19 Dec 2016 16:07:03 +0200 Message-ID: Subject: Re: About Tombstones and TTLs To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=94eb2c081fde2c2a570544036fdf archived-at: Mon, 19 Dec 2016 14:07:19 -0000 --94eb2c081fde2c2a570544036fdf Content-Type: text/plain; charset=UTF-8 Thanks for the explanation Matija, but fortunately, that I know. Forgot to mention that I'm using a multi DC cluster. I'll try to summarize just the questions I have, because my email was indeed quite long :-) - Why setting gc_grace_seconds=0 will disable hints for the table? - How can an expired TTL record be deleted by Cassandra without tombstoning or compaction? Aren't SSTables immutable files, and expired records are removed through compaction? - If I only use TTL for deletion, do I still need gc_grace_seconds to be bigger than 0? - If I only use TTL for deletion, but use updates as well, do I need gc_grace_seconds to be bigger than 0? Thanks! Shalom Sagges DBA T: +972-74-700-4035 We Create Meaningful Connections On Mon, Dec 19, 2016 at 2:39 PM, Matija Gobec wrote: > Hi, > > gc_grace_seconds is used to maintain data consistency in some failure > scenarios. When manually deleting data that action creates tombstones which > are kept for that defined period before being compacted. If one of the > replica nodes is down while deleting data and it gets back up after the > gc_grace_seconds defined period your previously delete data will reappear > (ghost data). As it is stated in datastax documentation on a single node > you can set gc_grace_seconds to 0 and you can do the same for tables that > contain only data with TTL. In the mentioned failure scenario your downed > node will have data with TTL information and no data inconsistency will > happen. > > On Mon, Dec 19, 2016 at 1:00 PM, Shalom Sagges > wrote: > >> Hi Everyone, >> >> I was reading a blog on TWCS by Alex Dejanovski from The Last Pickle ( >> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html) >> >> When I got to the comments section, I didn't understand why setting >> gc_grace_seconds to 0 will disable hints for the associated table: >> *"It is a very good point that gc_grace_seconds shouldn't be lowered too >> much as its impact on hinted handoff is not a well known fact, and using a >> value of 0 will purely disable hints on the table."* >> >> When I tried to read some more about deletes and TTLs, I got to a >> Datastax documentation https://docs.datastax.com/en/cassandra/3.0/cas >> sandra/dml/dmlAboutDeletes.html >> stating the following: >> >> *Cassandra allows you to set a default_time_to_live property for an >> entire table. Columns and rows marked with regular TTLs are processed as >> described above; but when a record exceeds the table-level TTL, Cassandra >> deletes it immediately, without tombstoning or compaction.* >> >> Which got me a bit more confused. >> So I hope someone can shed some light on some questions I've got: >> >> >> - Why setting gc_grace_seconds=0 will disable hints for the table? >> - How can an expired TTL record be deleted by Cassandra without >> tombstoning or compaction? Aren't SSTables immutable files, and expired >> records are removed through compaction? >> - If I only use TTL for deletion, do I still need gc_grace_seconds to >> be bigger than 0? >> - If I only use TTL for deletion, but use updates as well, do I need >> gc_grace_seconds to be bigger than 0? >> >> >> Sorry for all those questions, I'm just really confused from all the >> TTL/tombstones subject (still a newbie). >> >> Thanks a lot! >> >> >> Shalom Sagges >> DBA >> T: +972-74-700-4035 <+972%2074-700-4035> >> >> We Create Meaningful Connections >> >> >> >> This message may contain confidential and/or privileged information. >> If you are not the addressee or authorized to receive this on behalf of >> the addressee you must not use, copy, disclose or take action based on this >> message or any information herein. >> If you have received this message in error, please advise the sender >> immediately by reply email and delete this message. Thank you. >> > > -- This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply email and delete this message. Thank you. --94eb2c081fde2c2a570544036fdf Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for the explanation Matija, but fortunately, that I= know. Forgot to mention that I'm using a multi DC cluster.=C2=A0
I= 'll try to summarize just the questions I have, because my email was in= deed quite long :-)

  • Why setting gc_grace_seconds=3D0 will disab= le hints for the table?
  • How can an expir= ed TTL record be deleted by Cassandra without tombstoning or compaction? Ar= en't SSTables immutable files, and expired records are removed through = compaction?
  • If I only use TTL for deleti= on, do I still need gc_grace_seconds to be bigger than 0?
  • If I only use TTL for deletion, but use updates as well, = do I need gc_grace_seconds to be bigger than 0?

Thanks!=C2=A0

=
=C2=A0
Shalom Sagges
DBA
T: +972-74-700-4035
We Cr= eate Meaningful Connections

=C2=A0

On Mon, Dec 19, 2016 at 2:39 PM, Matija Gobe= c <matija0204@gmail.com> wrote:
Hi,

gc_grace_seconds is used t= o maintain data consistency in some failure scenarios. When manually deleti= ng data that action creates tombstones which are kept for that defined peri= od before being compacted. If one of the replica nodes is down while deleti= ng data and it gets back up after the gc_grace_seconds defined period your = previously delete data will reappear (ghost data). As it is stated in datas= tax documentation on a single node you can set gc_grace_seconds to 0 and yo= u can do the same for tables that contain only data with TTL. In the mentio= ned failure scenario your downed node will have data with TTL information a= nd no data inconsistency will happen.

On Mon, Dec 19, 2016= at 1:00 PM, Shalom Sagges <shaloms@liveperson.com> wro= te:
Hi Everyone,=C2=A0

I was reading a blog o= n TWCS by=C2=A0Alex Dejanovski from The Last Pickle (http://the= lastpickle.com/blog/2016/12/08/TWCS-part1.html)=C2=A0
When I got to the comments section, I didn't understand wh= y setting gc_grace_seconds to 0 will disable hints for the associated table= :=C2=A0
"It is a very good point that gc_grace_seconds sh= ouldn't be lowered too much as its impact on hinted handoff is not a we= ll known fact, and using a value of 0 will purely disable hints on the tabl= e."

When I tried to read some = more about deletes and TTLs, I got to a Datastax documentation=C2=A0https://docs.datastax.com/en/cassandra/3.0/c= assandra/dml/dmlAboutDeletes.html
stating the follo= wing:

Cassandra allows you to set a default_tim= e_to_live property for an entire table. Columns and rows marked with regula= r TTLs are processed as described above; but when a record exceeds the tabl= e-level TTL, Cassandra deletes it immediately, without tombstoning or compa= ction.

Which got me a bit more conf= used.=C2=A0
So I hope someone can shed some light on some questio= ns I've got:

  • Why setting gc_grace_seco= nds=3D0 will disable hints for the table?
  • How can an expired TTL re= cord be deleted by Cassandra without tombstoning or compaction? Aren't = SSTables immutable files, and expired records are removed through compactio= n?
  • If I only use TTL for deletion, do I still need gc_grace_seconds= to be bigger than 0?
  • If I only use TTL for deletion, but use updat= es as well, do I need gc_grace_seconds to be bigger than 0?

Sorry for all those questions, I'm just really = confused from all the TTL/tombstones subject (still a newbie). =C2=A0
=
=C2=A0
Thanks a lot!

=C2=A0
Shalom Sagges<= /td>
DBA
T: +972-74-700-4035
We Create Meaningful Connections

=C2=A0

This message may contain confidential and= /or privileged information.=C2=A0
If you = are not the addressee or authorized to receive this on behalf of the addres= see you must not use, copy, disclose or take action based on this message o= r any information herein.=C2=A0
If you ha= ve received this message in error, please advise the sender immediately by = reply email and delete this message. Thank you.



This message may contain confidential and/or privileg= ed information.=C2=A0
If you are not the = addressee or authorized to receive this on behalf of the addressee you must= not use, copy, disclose or take action based on this message or any inform= ation herein.=C2=A0
If you have received = this message in error, please advise the sender immediately by reply email = and delete this message. Thank you.
--94eb2c081fde2c2a570544036fdf--