incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kais Ahmed <k...@neteck-fr.com>
Subject Re: Cassandra read reapair
Date Wed, 29 May 2013 16:51:38 GMT
Thanks aaron


2013/5/28 aaron morton <aaron@thelastpickle.com>

> Start using QUOURM for reads and writes and then run a nodetool repair.
>
> That should get you back to the land of the consistent.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Consultant
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 27/05/2013, at 10:01 PM, Kais Ahmed <kais@neteck-fr.com> wrote:
>
> Hi aaron,
>
> I was sure that  phpcassa use QUORUM for W and R by default, but you're
> right, the default CL for R AND W is ONE.
>
> We are in this configuration W + R < N, how can i do to repair some keys
> that always return inconsistent data.
>
> Thanks,
>
>
>
>
>
>
>
>
>
> 2013/5/24 Kais Ahmed <kais@neteck-fr.com>
>
>> Hi aaron an thanks,
>>
>>
>> > If you are reading and writing at CL QUOURM and getting inconsistent
>> results that sounds like a bug. If you are mixing the CL levels such that R
>> + W <= N then it's expected behaviour.
>> I think it's a bug, it concern only some keys (~200 over 120 000 keys) on
>> one column family,
>>
>> As I understand it, if  R+W <= N it's expected behaviour at a moment,
>> but read repair will correct the inconsistent data (related to
>> read_repair_chance value)
>> and the next query will return a consistent data , right ?
>>
>> Here is an exemple of a result that i have on a key (keyspace RF 3), 3
>> differents replicas :
>>
>> [default@prod] ASSUME contacts_timeordered KEYS AS ascii;
>>
>> [default@prod] get contacts_timeordered['1425185-IGNORED'];
>> => (column=1a927740-97ec-11e2-ab16-a1afd66a735e, value=363838353936,
>> timestamp=1364505108842098)
>> => (column=1a93d5b0-97ec-11e2-888c-2bf068e0f754, value=31373930303330,
>> timestamp=1364505108851088)
>> => (column=b5c559c0-9d0f-11e2-8682-f7ecd4112689, value=32343130303930,
>> timestamp=1365070157421869)
>> => (column=7ba22b90-b48b-11e2-a2c2-914573921d9f, value=32353031353039,
>> timestamp=1367652194221857)
>> => (column=63ef5d80-b7e8-11e2-abf8-593c289227cd, value=32383435323830,
>> timestamp=1368021951146575)
>> => (column=d6383fc0-b810-11e2-a880-bd2ecacbaee3, value=31363334363737,
>> timestamp=1368039322753824)
>> => (column=f47d8e60-bd3f-11e2-88f4-533a93fe9432, value=32373938313038,
>> timestamp=1368609315699785)
>> => (column=c5bfe060-bf8e-11e2-ab1f-07be407aff58, value=32333634353034,
>> timestamp=1368863069848610)
>> => (column=f07ae4b0-c42f-11e2-8064-9794e872eb2b, value=363838353936,
>> timestamp=1369372095163129)
>> Returned 9 results.
>> Elapsed time: 10 msec(s).
>>
>> [default@prod] get contacts_timeordered['1425185-IGNORED'];
>> => (column=b5c559c0-9d0f-11e2-8682-f7ecd4112689, value=32343130303930,
>> timestamp=1365070157421869)
>> => (column=7ba22b90-b48b-11e2-a2c2-914573921d9f, value=32353031353039,
>> timestamp=1367652194221857)
>> => (column=63ef5d80-b7e8-11e2-abf8-593c289227cd, value=32383435323830,
>> timestamp=1368021951146575)
>> => (column=d6383fc0-b810-11e2-a880-bd2ecacbaee3, value=31363334363737,
>> timestamp=1368039322753824)
>> => (column=f47d8e60-bd3f-11e2-88f4-533a93fe9432, value=32373938313038,
>> timestamp=1368609315699785)
>> => (column=c5bfe060-bf8e-11e2-ab1f-07be407aff58, value=32333634353034,
>> timestamp=1368863069848610)
>> => (column=f07ae4b0-c42f-11e2-8064-9794e872eb2b, value=363838353936,
>> timestamp=1369372095163129)
>> Returned 7 results.
>> Elapsed time: 7.49 msec(s).
>>
>> [default@prod] get contacts_timeordered['1425185-IGNORED'];
>> => (column=1a93d5b0-97ec-11e2-888c-2bf068e0f754, value=31373930303330,
>> timestamp=1364505108851088)
>> => (column=b5c559c0-9d0f-11e2-8682-f7ecd4112689, value=32343130303930,
>> timestamp=1365070157421869)
>> => (column=7ba22b90-b48b-11e2-a2c2-914573921d9f, value=32353031353039,
>> timestamp=1367652194221857)
>> => (column=63ef5d80-b7e8-11e2-abf8-593c289227cd, value=32383435323830,
>> timestamp=1368021951146575)
>> => (column=d6383fc0-b810-11e2-a880-bd2ecacbaee3, value=31363334363737,
>> timestamp=1368039322753824)
>> => (column=f47d8e60-bd3f-11e2-88f4-533a93fe9432, value=32373938313038,
>> timestamp=1368609315699785)
>> => (column=c5bfe060-bf8e-11e2-ab1f-07be407aff58, value=32333634353034,
>> timestamp=1368863069848610)
>> => (column=f07ae4b0-c42f-11e2-8064-9794e872eb2b, value=363838353936,
>> timestamp=1369372095163129)
>> Returned 8 results.
>> Elapsed time: 9.37 msec(s).
>>
>> Do I have to change read_repair_chance to 1 to correct the inconsistency,
>> nodetool repair don't solve it.
>>
>> Thanks a lot,
>>
>>
>>
>>
>>
>> 2013/5/23 aaron morton <aaron@thelastpickle.com>
>>
>>> If you are reading and writing at CL QUOURM and getting inconsistent
>>> results that sounds like a bug. If you are mixing the CL levels such that R
>>> + W <= N then it's expected behaviour.
>>>
>>>
>>> Can you reproduce the issue outside of your app ?
>>>
>>> Cheers
>>>
>>>    -----------------
>>> Aaron Morton
>>> Freelance Cassandra Consultant
>>> New Zealand
>>>
>>> @aaronmorton
>>> http://www.thelastpickle.com
>>>
>>> On 21/05/2013, at 8:55 PM, Kais Ahmed <kais@neteck-fr.com> wrote:
>>>
>>> > Checking you do not mean the row key is corrupt and cannot be read.
>>> Yes, i can read it but all read don't return the same result except for
>>> CL ALL
>>>
>>> > By default in 1.X and beyond the default read repair chance is 0.1,
>>> so it's only enabled on 10% of requests.
>>> You are right read repair chance is set to 0.1, but i launched a read
>>> repair which did not solved the problem. Any idea?
>>>
>>> >What CL are you writing at ?
>>> All write are in CL QUORUM
>>>
>>> thank you aaron for your answer.
>>>
>>>
>>> 2013/5/21 aaron morton <aaron@thelastpickle.com>
>>>
>>>> Only some keys of one CF are corrupt.
>>>>
>>>> Checking you do not mean the row key is corrupt and cannot be read.
>>>>
>>>> I thought using CF ALL, would correct the problem with READ REPAIR, but
>>>>  by returning to CL QUORUM, the problem persists.
>>>>
>>>> By default in 1.X and beyond the default read repair chance is 0.1, so
>>>> it's only enabled on 10% of requests.
>>>>
>>>> In the absence of further writes all reads (at any CL) should return
>>>> the same value.
>>>>
>>>> What CL are you writing at ?
>>>>
>>>> Cheers
>>>>
>>>>    -----------------
>>>> Aaron Morton
>>>> Freelance Cassandra Consultant
>>>> New Zealand
>>>>
>>>> @aaronmorton
>>>> http://www.thelastpickle.com
>>>>
>>>> On 19/05/2013, at 1:28 AM, Kais Ahmed <kais@neteck-fr.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I encountered a consistency problem one some keys using phpcassa and
>>>> Cassandra 1.2.3 since a server crash
>>>>
>>>> Only some keys of one CF are corrupt.
>>>>
>>>> I lauched a nodetool repair that successfully completed but don't
>>>> correct the issue.
>>>>
>>>>
>>>>
>>>> When i try to get a corrupt Key with :
>>>>
>>>> CL ONE, the result contains 7 or 8 or 9 columns
>>>>
>>>> CL QUORUM, result contains 8 or 9 columns
>>>>
>>>> CL ALL, the data is consistent and returns always 9 columns
>>>>
>>>>
>>>> I thought using CF ALL, would correct the problem with READ REPAIR, but by
>>>> returning to CL QUORUM, the problem persists.
>>>>
>>>>
>>>> Thank you for your help
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
>

Mime
View raw message