Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of kohlisankalp@gmail.com
 designates 209.85.216.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJV_UYfNo1uM46NvU-mcsqo-gdULqYLd4LXZo8yuONktBOJnAg@mail.gmail.com>
References: 
 <CA+VSrLo=3unnfjxnpXDKW7Hh3coFyhOAsqzRmFVENG4LOSuvLg@mail.gmail.com>
 <CA+VSrLrn0t1zvPCKPU8RDDEeqVsdewb-8_yQ9hrPnXQWno_OEg@mail.gmail.com>
 <CAJV_UYfNo1uM46NvU-mcsqo-gdULqYLd4LXZo8yuONktBOJnAg@mail.gmail.com>
From: sankalp kohli <kohlisankalp@gmail.com>
Date: Mon, 26 Aug 2013 12:09:38 -0700
Message-ID: 
 <CAEPxca0QOLhFwxaOmgEb4jTRCsrJ=tdZmxCN0-TVsN-PXuY4rQ@mail.gmail.com>
Subject: Re: Periodical deletes and compaction strategy
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=047d7bf16090d31fb204e4de815d

--047d7bf16090d31fb204e4de815d
Content-Type: text/plain; charset=ISO-8859-1

The problem is that tombstones will hang in there till GC grace period. You
can reduce the GC grace period and then catch lost deletes in the
application layer if you know you should not be seeing such an old record.

Also in 1.2, they have some setting which enable an sstable to be compacted
if it has lot of tombstones. You might want to look at that.


On Mon, Aug 26, 2013 at 3:31 AM, cem <cayiroglu@gmail.com> wrote:

> Hi Alain,
>
> I solved the same issue by implementing a client that manages time range
> partitions. Each time range partition is a CF.
>
> Cem.
>
>
> On Mon, Aug 26, 2013 at 11:34 AM, Alain RODRIGUEZ <arodrime@gmail.com>wrote:
>
>> Hi,
>>
>> Any guidance on this topic would be appreciated :).
>>
>>
>> 2013/8/23 Alain RODRIGUEZ <arodrime@gmail.com>
>>
>>> Hi,
>>>
>>> I am currently using about 10 CF to store temporal data. Those data are
>>> growing pretty big (hundreds of GB when I actually only need information
>>> from the last month - i.e. about hundreds of MB).
>>>
>>> I am going to delete old (and useless) data, I cannot always use TTL
>>> since I have counters too. Yet I know that deletes are a bit tricky in
>>> Cassandra, due to the fact that they are distributed.
>>>
>>> I was wondering about the best way to keep high performance and get rid
>>> of tombstones easily.
>>>
>>> I was considering 2 ways to do it :
>>>
>>> - Major compaction on these 10 CF to force them to always keep fresh
>>> data only and remove tombstones
>>> - LCS to have more chance to get all parts of the row in one SSTable,
>>> allowing tombstones to be removed eventually.
>>>
>>> What would be the better option (i.e. what would be the impact of both
>>> solutions) ?
>>> Do you need more information about those CF to answer this question ?
>>>
>>> Any insight is welcome, as always.
>>>
>>> Alain
>>>
>>
>>
>

--047d7bf16090d31fb204e4de815d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">The problem is that tombstones will hang in there till GC =
grace period. You can reduce the GC grace period and then catch lost delete=
s in the application layer if you know you should not be seeing such an old=
 record.=A0<div>

<br></div><div style>Also in 1.2, they have some setting which enable an ss=
table to be compacted if it has lot of tombstones. You might want to look a=
t that.=A0</div></div><div class=3D"gmail_extra"><br><br><div class=3D"gmai=
l_quote">

On Mon, Aug 26, 2013 at 3:31 AM, cem <span dir=3D"ltr">&lt;<a href=3D"mailt=
o:cayiroglu@gmail.com" target=3D"_blank">cayiroglu@gmail.com</a>&gt;</span>=
 wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bor=
der-left:1px #ccc solid;padding-left:1ex">

<div dir=3D"ltr">Hi Alain,<div><br></div><div>I solved the same issue by im=
plementing a client that manages time range partitions. Each time range par=
tition is a CF.</div><span class=3D"HOEnZb"><font color=3D"#888888"><div><b=
r>

</div><div>Cem.<br></div></font></span></div><div class=3D"HOEnZb"><div cla=
ss=3D"h5"><div class=3D"gmail_extra">
<br><br><div class=3D"gmail_quote">On Mon, Aug 26, 2013 at 11:34 AM, Alain =
RODRIGUEZ <span dir=3D"ltr">&lt;<a href=3D"mailto:arodrime@gmail.com" targe=
t=3D"_blank">arodrime@gmail.com</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">


<div dir=3D"ltr"><div>Hi,</div><div><br></div>Any guidance on this topic wo=
uld be appreciated :).<br></div><div><div><div class=3D"gmail_extra"><br><b=
r><div class=3D"gmail_quote">2013/8/23 Alain RODRIGUEZ <span dir=3D"ltr">&l=
t;<a href=3D"mailto:arodrime@gmail.com" target=3D"_blank">arodrime@gmail.co=
m</a>&gt;</span><br>


<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div>Hi,</div><div><br></di=
v><div>I am currently using about 10 CF to store temporal data. Those data =
are growing pretty big (hundreds of GB when I actually only need informatio=
n from the last month - i.e. about hundreds of MB).</div>


<div><br></div><div>I am going to delete old (and useless) data, I cannot a=
lways use TTL since I have counters too. Yet I know that deletes are a bit =
tricky in Cassandra, due to the fact that they are distributed.</div><div>


<br></div><div>I was wondering about the best way to keep high performance =
and get rid of tombstones easily.</div><div><br></div><div>I was considerin=
g 2 ways to do it :</div><div><br></div><div>- Major compaction on these 10=
 CF to force them to always keep fresh data only and remove tombstones</div=
>


<div>- LCS to have more chance to get all parts of the row in one SSTable, =
allowing tombstones to be removed eventually.</div><div><br></div><div>What=
 would be the better option (i.e. what would be the impact of both solution=
s) ?</div>


<div>Do you need more information about those CF to answer this question ?<=
/div><div><br></div><div>Any insight is welcome, as always.</div><span><fon=
t color=3D"#888888"><div><br></div><div>Alain</div></font></span></div>


</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--047d7bf16090d31fb204e4de815d--