incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <arodr...@gmail.com>
Subject Re: No deletes - is periodic repair needed? I think not...
Date Thu, 06 Feb 2014 11:02:26 GMT
Hi,

In a distributed system, such as Cassandra, things can happen (node down,
stop the world GC, hardware issue, ...) and desynchronize replicas, isn't
repairing also a needed operation to keep replicas up to date at least once
a week or once a month ? It is a strong and reliable process to keep things
synced, isn't it ?

I know that read repairs and hinted handoff are also there to handle this
kind of issues, but they might fail (I saw a lot of error in the logs
around hints not being delivered - some people even disable them - and read
repairs are often configured to trigger on 10% of the reads).


2014-01-28 14:53 GMT+01:00 Sylvain Lebresne <sylvain@datastax.com>:

>
>> I have actually set up one of our application streams such that the same
>> key is only overwritten with a monotonically increasing ttl.
>>
>> For example, a breaking news item might have an initial ttl of 60
>> seconds, followed in 45 seconds by an update with a ttl of 3000 seconds,
>> followed by an 'ignore me' update in 600 seconds with a ttl of 30 days (our
>> maximum ttl) when the article is published.
>>
>> My understanding is that this case fits the criteria and no 'periodic
>> repair' is needed.
>>
>
> That's correct. The real criteria for not needing repair if you do no
> deletes but only TTL is "update only with monotonically increasing (non
> necessarily strictly) ttl". Always setting the same TTL is just a special
> case of that, but it's the most commonly used one I think, so I tend to
> simplify it to that case.
>
>
>>
>> I guess another thing I would point out that is easy to miss or forget
>> (if you are a newish user like me), is that ttl's are fine-grained, by
>> column. So we are talking 'fixed' or 'variable' by individual column, not
>> by table. Which means, in my case, that ttl's can vary widely across a
>> table, but as long as I constrain them by key value to be fixed or
>> monotonically increasing, it fits the criteria.
>>
>
> We're talking monotonically increasing ttl "for a given primary key' if
> we're talking the CQL language and "for a given column" if we're talking
> the thrift one. Not "by table".
>
> --
> Sylvain
>
>
>
>>
>> Cheers,
>>
>> Michael
>>
>>
>> On Tue, Jan 28, 2014 at 4:18 AM, Sylvain Lebresne <sylvain@datastax.com>wrote:
>>
>>> On Tue, Jan 28, 2014 at 1:05 AM, Edward Capriolo <edlinuxguru@gmail.com>wrote:
>>>
>>>> If you have only ttl columns, and you never update the column I would
>>>> not think you need a repair.
>>>>
>>>
>>> Right, no deletes and no updates is the case 1. of Michael on which I
>>> think we all agree 'periodic repair to avoid resurrected columns' is not
>>> required.
>>>
>>>
>>>>
>>>> Repair cures lost deletes. If all your writes have a ttl a lost write
>>>> should not matter since the column was never written to the node and thus
>>>> could never be resurected on said node.
>>>>
>>>
>>>  I'm sure we're all in agreement here, but for the record, this is only
>>> true if you have no updates (overwrites) and/or if all writes have the
>>> *same* ttl. Because in the general case, a column with a relatively short
>>> TTL is basically very close to a delete, while a column with a long TTL is
>>> very close from one that has no TTL. If the former column (with short TTL)
>>> overwrites the latter one (with long TTL), and if one nodes misses the
>>> overwrite, that node could resurrect the column with the longer TTL (until
>>> that column expires that is). Hence the separation of the case 2. (fixed
>>> ttl, no repair needed) and 2.a. (variable ttl, repair may be needed).
>>>
>>> --
>>> Sylvain
>>>
>>>
>>>>
>>>> Unless i am missing something.
>>>>
>>>> On Monday, January 27, 2014, Laing, Michael <michael.laing@nytimes.com>
>>>> wrote:
>>>> > Thanks Sylvain,
>>>> > Your assumption is correct!
>>>> > So I think I actually have 4 classes:
>>>> > 1.    Regular values, no deletes, no overwrites, write heavy,
>>>> variable ttl's to manage size
>>>> > 2.    Regular values, no deletes, some overwrites, read heavy (10 to
>>>> 1), fixed ttl's to manage size
>>>> > 2.a. Regular values, no deletes, some overwrites, read heavy (10 to
>>>> 1), variable ttl's to manage size
>>>> > 3.    Counter values, no deletes, update heavy, rotation/truncation
>>>> to manage size
>>>> > Only 2.a. above requires me to do 'periodic repair'.
>>>> > What I will actually do is change my schema and applications slightly
>>>> to eliminate the need for overwrites on the only table I have in that
>>>> category.
>>>> > And I will set gc_grace_period to 0 for the tables in the updated
>>>> schema and drop 'periodic repair' from the schedule.
>>>> > Cheers,
>>>> > Michael
>>>> >
>>>> >
>>>> > On Mon, Jan 27, 2014 at 4:22 AM, Sylvain Lebresne <
>>>> sylvain@datastax.com> wrote:
>>>> >>
>>>> >> By periodic repair, I'll assume you mean "having to run repair every
>>>> gc_grace period to make sure no deleted entries resurrect". With that
>>>> assumption:
>>>> >>
>>>> >>>
>>>> >>> 1. Regular values, no deletes, no overwrites, write heavy, ttl's
to
>>>> manage size
>>>> >>
>>>> >> Since 'repair within gc_grace' is about avoiding value that have
>>>> been deleted to resurrect, if you do no delete nor overwrites, you're in
no
>>>> risk of that (and don't need to 'repair withing gc_grace').
>>>> >>
>>>> >>>
>>>> >>> 2. Regular values, no deletes, some overwrites, read heavy (10
to
>>>> 1), ttl's to manage size
>>>> >>
>>>> >> It depends a bit. In general, if you always set the exact same TTL
>>>> on every insert (implying you always set a TTL), then you have nothing to
>>>> worry about. If the TTL varies (of if you only set TTL some of the times),
>>>> then you might still need to have some periodic repairs. That being said,
>>>> if there is no deletes but only TTLs, then the TTL kind of lengthen the
>>>> period at which you need to do repair: instead of needing to repair withing
>>>> gc_grace, you only need to repair every gc_grace + min(TTL) (where min(TTL)
>>>> is the smallest TTL you set on columns).
>>>> >>>
>>>> >>> 3. Counter values, no deletes, update heavy, rotation/truncation
to
>>>> manage size
>>>> >>
>>>> >> No deletes and no TTL implies that your fine (as in, there is no
>>>> need for 'repair withing gc_grace').
>>>> >>
>>>> >> --
>>>> >> Sylvain
>>>> >
>>>>
>>>> --
>>>> Sorry this was sent from mobile. Will do less grammar and spell check
>>>> than usual.
>>>>
>>>
>>>
>>
>

Mime
View raw message