incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kais Ahmed <k...@neteck-fr.com>
Subject Re: Cassandra read reapair
Date Fri, 24 May 2013 16:58:59 GMT
Hi aaron an thanks,

> If you are reading and writing at CL QUOURM and getting inconsistent
results that sounds like a bug. If you are mixing the CL levels such that R
+ W <= N then it's expected behaviour.
I think it's a bug, it concern only some keys (~200 over 120 000 keys) on
one column family,

As I understand it, if  R+W <= N it's expected behaviour at a moment, but
read repair will correct the inconsistent data (related to
read_repair_chance value)
and the next query will return a consistent data , right ?

Here is an exemple of a result that i have on a key (keyspace RF 3), 3
differents replicas :

[default@prod] ASSUME contacts_timeordered KEYS AS ascii;

[default@prod] get contacts_timeordered['1425185-IGNORED'];
=> (column=1a927740-97ec-11e2-ab16-a1afd66a735e, value=363838353936,
timestamp=1364505108842098)
=> (column=1a93d5b0-97ec-11e2-888c-2bf068e0f754, value=31373930303330,
timestamp=1364505108851088)
=> (column=b5c559c0-9d0f-11e2-8682-f7ecd4112689, value=32343130303930,
timestamp=1365070157421869)
=> (column=7ba22b90-b48b-11e2-a2c2-914573921d9f, value=32353031353039,
timestamp=1367652194221857)
=> (column=63ef5d80-b7e8-11e2-abf8-593c289227cd, value=32383435323830,
timestamp=1368021951146575)
=> (column=d6383fc0-b810-11e2-a880-bd2ecacbaee3, value=31363334363737,
timestamp=1368039322753824)
=> (column=f47d8e60-bd3f-11e2-88f4-533a93fe9432, value=32373938313038,
timestamp=1368609315699785)
=> (column=c5bfe060-bf8e-11e2-ab1f-07be407aff58, value=32333634353034,
timestamp=1368863069848610)
=> (column=f07ae4b0-c42f-11e2-8064-9794e872eb2b, value=363838353936,
timestamp=1369372095163129)
Returned 9 results.
Elapsed time: 10 msec(s).

[default@prod] get contacts_timeordered['1425185-IGNORED'];
=> (column=b5c559c0-9d0f-11e2-8682-f7ecd4112689, value=32343130303930,
timestamp=1365070157421869)
=> (column=7ba22b90-b48b-11e2-a2c2-914573921d9f, value=32353031353039,
timestamp=1367652194221857)
=> (column=63ef5d80-b7e8-11e2-abf8-593c289227cd, value=32383435323830,
timestamp=1368021951146575)
=> (column=d6383fc0-b810-11e2-a880-bd2ecacbaee3, value=31363334363737,
timestamp=1368039322753824)
=> (column=f47d8e60-bd3f-11e2-88f4-533a93fe9432, value=32373938313038,
timestamp=1368609315699785)
=> (column=c5bfe060-bf8e-11e2-ab1f-07be407aff58, value=32333634353034,
timestamp=1368863069848610)
=> (column=f07ae4b0-c42f-11e2-8064-9794e872eb2b, value=363838353936,
timestamp=1369372095163129)
Returned 7 results.
Elapsed time: 7.49 msec(s).

[default@prod] get contacts_timeordered['1425185-IGNORED'];
=> (column=1a93d5b0-97ec-11e2-888c-2bf068e0f754, value=31373930303330,
timestamp=1364505108851088)
=> (column=b5c559c0-9d0f-11e2-8682-f7ecd4112689, value=32343130303930,
timestamp=1365070157421869)
=> (column=7ba22b90-b48b-11e2-a2c2-914573921d9f, value=32353031353039,
timestamp=1367652194221857)
=> (column=63ef5d80-b7e8-11e2-abf8-593c289227cd, value=32383435323830,
timestamp=1368021951146575)
=> (column=d6383fc0-b810-11e2-a880-bd2ecacbaee3, value=31363334363737,
timestamp=1368039322753824)
=> (column=f47d8e60-bd3f-11e2-88f4-533a93fe9432, value=32373938313038,
timestamp=1368609315699785)
=> (column=c5bfe060-bf8e-11e2-ab1f-07be407aff58, value=32333634353034,
timestamp=1368863069848610)
=> (column=f07ae4b0-c42f-11e2-8064-9794e872eb2b, value=363838353936,
timestamp=1369372095163129)
Returned 8 results.
Elapsed time: 9.37 msec(s).

Do I have to change read_repair_chance to 1 to correct the inconsistency,
nodetool repair don't solve it.

Thanks a lot,





2013/5/23 aaron morton <aaron@thelastpickle.com>

> If you are reading and writing at CL QUOURM and getting inconsistent
> results that sounds like a bug. If you are mixing the CL levels such that R
> + W <= N then it's expected behaviour.
>
>
> Can you reproduce the issue outside of your app ?
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Consultant
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 21/05/2013, at 8:55 PM, Kais Ahmed <kais@neteck-fr.com> wrote:
>
> > Checking you do not mean the row key is corrupt and cannot be read.
> Yes, i can read it but all read don't return the same result except for
> CL ALL
>
> > By default in 1.X and beyond the default read repair chance is 0.1, so
> it's only enabled on 10% of requests.
> You are right read repair chance is set to 0.1, but i launched a read
> repair which did not solved the problem. Any idea?
>
> >What CL are you writing at ?
> All write are in CL QUORUM
>
> thank you aaron for your answer.
>
>
> 2013/5/21 aaron morton <aaron@thelastpickle.com>
>
>> Only some keys of one CF are corrupt.
>>
>> Checking you do not mean the row key is corrupt and cannot be read.
>>
>> I thought using CF ALL, would correct the problem with READ REPAIR, but by
>> returning to CL QUORUM, the problem persists.
>>
>> By default in 1.X and beyond the default read repair chance is 0.1, so
>> it's only enabled on 10% of requests.
>>
>> In the absence of further writes all reads (at any CL) should return the
>> same value.
>>
>> What CL are you writing at ?
>>
>> Cheers
>>
>>    -----------------
>> Aaron Morton
>> Freelance Cassandra Consultant
>> New Zealand
>>
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 19/05/2013, at 1:28 AM, Kais Ahmed <kais@neteck-fr.com> wrote:
>>
>> Hi all,
>>
>> I encountered a consistency problem one some keys using phpcassa and
>> Cassandra 1.2.3 since a server crash
>>
>> Only some keys of one CF are corrupt.
>>
>> I lauched a nodetool repair that successfully completed but don't correct
>> the issue.
>>
>>
>>
>> When i try to get a corrupt Key with :
>>
>> CL ONE, the result contains 7 or 8 or 9 columns
>>
>> CL QUORUM, result contains 8 or 9 columns
>>
>> CL ALL, the data is consistent and returns always 9 columns
>>
>>
>> I thought using CF ALL, would correct the problem with READ REPAIR, but by
>> returning to CL QUORUM, the problem persists.
>>
>>
>> Thank you for your help
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>

Mime
View raw message