incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: iteration does not yield all data with consistency ONE
Date Mon, 15 Nov 2010 13:07:13 GMT
Great. Thanks for helping us track that down!

On Mon, Nov 15, 2010 at 6:45 AM, Eric van Orsouw
<eric.van.orsouw@eventis.nl> wrote:
> Jonathan, I have downloaded 0.6.8 and the iteration problem is fixed.
> Thanks,
> Eric
>
> -----Original Message-----
> From: Jonathan Ellis [mailto:jbellis@gmail.com]
> Sent: donderdag 11 november 2010 23:05
> To: user
> Subject: Re: iteration does not yield all data with consistency ONE
>
> Can you try with the latest 0.6 svn branch?
> https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.6  It
> may be caused by https://issues.apache.org/jira/browse/CASSANDRA-1722
> which is fixed there.
>
> On Thu, Nov 11, 2010 at 4:56 AM, Eric van Orsouw
> <eric.van.orsouw@eventis.nl> wrote:
>> I have tried it with RF=1 and 2 nodes and the result was the same.
>> In this case I am not able to query anything, regardless the consistency level.
>>
>> I have conducted some more tests with 4 nodes and RF=2 and before each test I also
completely cleared all datafiles (data/commit/saved_cache).
>> Each time I do this the nodes get different locations on the ring and the noticed
behavior is different (but similar) each time.
>> I have added the details for 2 concrete scenario;
>>
>> ----
>> 1) clear all 3 data directories for all nodes
>> 2) startup all nodes; the ring is;
>>
>> # bin/nodetool -h 172.16.21.12 ring
>> Address       Status     Load          Range                  
                   Ring
>>                                       81296677393575658129031223607666487650
>> 172.16.21.12  Up         489 bytes     61644956546475371096781179489550878842
    |<--|
>> 172.16.21.14  Up         489 bytes     63811055530446251457866768401307252614
    |   |
>> 172.16.21.13  Up         489 bytes     71622328744148061175473750202165524046
    |   |
>> 172.16.21.11  Up         489 bytes     81296677393575658129031223607666487650
    |-->|
>>
>> 3) I use the following ColumnFamily     <ColumnFamily Name="Customers" ColumnType="Super"
CompareWith="AsciiType"/>
>> And add the following rows (key and values utf8 strings)
>> Customer['1'].simple.id='1'
>> Customer['2'].simple.id='2'
>> Customer['3'].simple.id='3'
>> Customer['4'].simple.id='4'
>>
>> 4) Iterating over the columns now yield;
>>        Iterating Nodes .11 thru.13 ==> all rows, regardless of consistency
level
>>        Iterating Node.14           ==> only row '3' and '4' with consistency
level ONE, all rows with consistency level All
>>
>> ----
>> Repating all process steps again yields the following ring and iteration results
>> # bin/nodetool -h 172.16.21.11 ring
>> Address       Status     Load          Range                  
                   Ring
>>                                       157208721428268855331814529932509412075
>> 172.16.21.11  Up         489 bytes     18662665007498667259581716616851966032
    |<--|
>> 172.16.21.13  Up         489 bytes     125997322478065775200568923383951796531
   |   |
>> 172.16.21.12  Up         489 bytes     148125235552363592916130178190645177045
   |   |
>> 172.16.21.14  Up         489 bytes     157208721428268855331814529932509412075
   |-->|
>>
>>        Iterating Nodes .11 .12 .14 ==> no data with consistency level One,
all rows with consistency level All
>>        Iterating Node .13          ==> only row '3' and '4' with consistency
level ONE, all rows with consistency level All
>>
>> I appears to me there is an inconsistency somewhere, I hope this info helps.
>>
>> Eric
>>
>>
>> -----Original Message-----
>> From: Jonathan Ellis [mailto:jbellis@gmail.com]
>> Sent: woensdag 10 november 2010 17:48
>> To: user
>> Subject: Re: iteration does not yield all data with consistency ONE
>>
>> Interesting.  Does it simplify further to RF=1 and 2 nodes?
>>
>> On Wed, Nov 10, 2010 at 8:58 AM, Eric van Orsouw
>> <eric.van.orsouw@eventis.nl> wrote:
>>> No, all nodes were up and running while the single key was inserted.
>>> The insert however was with consistency One. I assume however that the replicas
are still written in this case.
>>> It is btw also very reproducible.
>>>
>>> -----Original Message-----
>>> From: Jonathan Ellis [mailto:jbellis@gmail.com]
>>> Sent: woensdag 10 november 2010 15:44
>>> To: user
>>> Subject: Re: iteration does not yield all data with consistency ONE
>>>
>>> Was the node that should have the other replica of this row down when
>>> it was inserted?
>>>
>>> On Wed, Nov 10, 2010 at 6:08 AM, Eric van Orsouw
>>> <eric.van.orsouw@eventis.nl> wrote:
>>>>
>>>> Hello,
>>>>
>>>>
>>>>
>>>> We have a cluster of 4 nodes (0.6.6) and use the random partitioner and a
replication of 2.
>>>>
>>>> When I insert a number of rows I can always retrieve them by their explicit
id (get_range_slices("<key>","", 1).
>>>>
>>>> Playing with consistency levels and temporarily shutting down a Cassandra
node all yields the expected result.
>>>>
>>>>
>>>>
>>>> However when I use get_range_slices("","", n) to iterate over all rows, I
sometimes don't get anything (depending on the node).
>>>>
>>>>
>>>>
>>>> I then reduced the problem to inserting just a single row.
>>>>
>>>> Specifically, the 'iteration' only seems to succeed when I issue the request
to the node that contains the first copy.
>>>>
>>>> I Discovered that when I iterate using a consistency level of Quorum/All
the iteration always succeeds and I properly get the one row.
>>>>
>>>>
>>>>
>>>> So a solution would be to always use consistency level One/All but that has
a performance penalty.
>>>>
>>>>
>>>>
>>>> Can anyone explain why iterating using get_range_slices("","",n) does not
always function with consistency level One on all nodes?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Eric
>>>>
>>>>
>>>>
>>>> P.S. To rule out any discussion on whether or not to use iteration in the
first place, we only plan to use it for backup and periodic cleanup cycles.
>>>
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of Riptano, the source for professional Cassandra support
>>> http://riptano.com
>>>
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of Riptano, the source for professional Cassandra support
>> http://riptano.com
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Mime
View raw message