Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: local policy)
From: <mike.li@thomsonreuters.com>
To: <user@cassandra.apache.org>, <zli@voxeo.com>
Date: Sat, 3 Dec 2011 20:44:33 +0000
Subject: RE: [RELEASE] Apache Cassandra 1.0.5 released
Thread-Topic: [RELEASE] Apache Cassandra 1.0.5 released
Thread-Index: AcyxOMcnVdqqYlybQuy3HG0l19f9IwAw0Mzg
Message-ID: 
 <6F1D54DD1FE19B408496215469822A8701F0537D2B6E@OCDP-ERFMMBX03.ERF.THOMSON.COM>
References: 
 <CAKkz8Q3vMsk3i6L_pSKFA3+mtoVAoMa2Wo9dx8FouJif2PftbQ@mail.gmail.com>
 <000e01ccaf96$5f5ae790$1e10b6b0$@com>
 <CAAafH9Tv2HZ2zRJU_vF1F3rSprzm_hg2sAVo2PqnxztZBxJZ2A@mail.gmail.com>
 <003101ccafa1$29d03c50$7d70b4f0$@com>
 <CALdd-zhtnnyZAJxhFzxLQCxvzTo-PhczxCt3QKvnD8B5Nx+rLQ@mail.gmail.com>
 <004301ccb018$2af69ad0$80e3d070$@com>
 <CAKkz8Q1-8WoRmZLg2Vt_VDgAPZ5uROrZU9XFjW3TQ3fz0mTeog@mail.gmail.com>
 <007201ccb032$36ae2ea0$a40a8be0$@com>
 <CAG+AvmsFqxMQpdETEiwh9wjMzJfYZtuTwWfyjjqTOhsASqG3Vw@mail.gmail.com>
 <F5A602F5-0E76-4D1D-B643-F84D540A0405@voxeo.com>
 <CAHJAGstXJPRjd2mYyWYa7v-7JtXCBbbFrGQt4WgtFcN1pcD7rw@mail.gmail.com>
 <CALdd-zgp5nsPRrT187fmAzi9YRZso1KP7AgmBMJ1-9rO5k+0jA@mail.gmail.com>
 <3DDB1229-4809-413C-A15E-984AC74E4164@ecyrd.com>
 <24E9FF55-0D88-4F85-AD93-88C708DFD920@voxeo.com>
In-Reply-To: <24E9FF55-0D88-4F85-AD93-88C708DFD920@voxeo.com>
Accept-Language: en-US, en-GB
Content-Language: en-US
acceptlanguage: en-US, en-GB
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0

Hi Zhong Li,

When I used stress tool to test, I got:

Operation [15] retried 10 times - error inserting key 0015 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [37] retried 10 times - error inserting key 0037 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [17] retried 10 times - error inserting key 0017 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [40] retried 10 times - error inserting key 0040 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [28] retried 10 times - error inserting key 0028 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [2] retried 10 times - error inserting key 0002 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [13] retried 10 times - error inserting key 0013 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [29] retried 10 times - error inserting key 0029 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [30] retried 10 times - error inserting key 0030 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [23] retried 10 times - error inserting key 0023 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [21] retried 10 times - error inserting key 0021 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [42] retried 10 times - error inserting key 0042 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [0] retried 10 times - error inserting key 0000 ((InvalidRequestException): unconfigured columnfamily Standard1)

Operation [11] retried 10 times - error inserting key 0011 ((InvalidRequestException): unconfigured columnfamily Standard1)

0,0,0,NaN,0
END

Do I need to create the column family manually first, or it is created by the stress tool automatically? Why did I get the errors above? 

Thanks,
Mike 

-----Original Message-----
From: Zhong Li [mailto:zli@voxeo.com] 
Sent: Friday, December 02, 2011 3:24 PM
To: user@cassandra.apache.org
Subject: Re: [RELEASE] Apache Cassandra 1.0.5 released

I just tested with stress tool, it is reproducible and timeout happens always.

./stress -d <mynode address> -e QUORUM -l 3 -o RANGE_SLICE
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
0,0,0,NaN,10
0,0,0,NaN,20
0,0,0,NaN,30
0,0,0,NaN,40
0,0,0,NaN,50
0,0,0,NaN,60
0,0,0,NaN,70
0,0,0,NaN,80
0,0,0,NaN,91
0,0,0,NaN,101
Operation [2] retried 10 times - error on calling get_indexed_slices for range offset 0000002 ((TimedOutException))

Operation [46] retried 10 times - error on calling get_indexed_slices for range offset 0000046 ((TimedOutException))

Operation [47] retried 10 times - error on calling get_indexed_slices for range offset 0000047 ((TimedOutException))

Operation [24] retried 10 times - error on calling get_indexed_slices for range offset 0000024 ((TimedOutException))

Operation [45] retried 10 times - error on calling get_indexed_slices for range offset 0000045 ((TimedOutException))

Operation [48] retried 10 times - error on calling get_indexed_slices for range offset 0000048 ((TimedOutException))

Operation [5] retried 10 times - error on calling get_indexed_slices for range offset 0000005 ((TimedOutException))

Operation [28] retried 10 times - error on calling get_indexed_slices for range offset 0000028 ((TimedOutException))

Operation [0] retried 10 times - error on calling get_indexed_slices for range offset 0000000 ((TimedOutException))

Operation [23] retried 10 times - error on calling get_indexed_slices for range offset 0000023 ((TimedOutException))

Operation [32] retried 10 times - error on calling get_indexed_slices for range offset 0000032 ((TimedOutException))

Operation [36] retried 10 times - error on calling get_indexed_slices for range offset 0000036 ((TimedOutException))

Operation [16] retried 10 times - error on calling get_indexed_slices for range offset 0000016 ((TimedOutException))

Operation [6] retried 10 times - error on calling get_indexed_slices for range offset 0000006 ((TimedOutException))

Operation [9] retried 10 times - error on calling get_indexed_slices for range offset 0000009 ((TimedOutException))

Operation [11] retried 10 times - error on calling get_indexed_slices for range offset 0000011 ((TimedOutException))

Operation [25] retried 10 times - error on calling get_indexed_slices for range offset 0000025 ((TimedOutException))

Operation [26] retried 10 times - error on calling get_indexed_slices for range offset 0000026 ((TimedOutException))

Operation [43] retried 10 times - error on calling get_indexed_slices for range offset 0000043 ((TimedOutException))

Operation [27] retried 10 times - error on calling get_indexed_slices for range offset 0000027 ((TimedOutException))

Operation [33] retried 10 times - error on calling get_indexed_slices for range offset 0000033 ((TimedOutException))

Operation [44] retried 10 times - error on calling get_indexed_slices for range offset 0000044 ((TimedOutException))

Operation [19] retried 10 times - error on calling get_indexed_slices for range offset 0000019 ((TimedOutException))

Operation [35] retried 10 times - error on calling get_indexed_slices for range offset 0000035 ((TimedOutException))

Operation [34] retried 10 times - error on calling get_indexed_slices for range offset 0000034 ((TimedOutException))

Operation [30] retried 10 times - error on calling get_indexed_slices for range offset 0000030 ((TimedOutException))

Operation [14] retried 10 times - error on calling get_indexed_slices for range offset 0000014 ((TimedOutException))

Operation [7] retried 10 times - error on calling get_indexed_slices for range offset 0000007 ((TimedOutException))

Operation [20] retried 10 times - error on calling get_indexed_slices for range offset 0000020 ((TimedOutException))

Operation [3] retried 10 times - error on calling get_indexed_slices for range offset 0000003 ((TimedOutException))

Operation [8] retried 10 times - error on calling get_indexed_slices for range offset 0000008 ((TimedOutException))

Operation [15] retried 10 times - error on calling get_indexed_slices for range offset 0000015 ((TimedOutException))

Operation [29] retried 10 times - error on calling get_indexed_slices for range offset 0000029 ((TimedOutException))

Operation [12] retried 10 times - error on calling get_indexed_slices for range offset 0000012 ((TimedOutException))

Operation [40] retried 10 times - error on calling get_indexed_slices for range offset 0000040 ((TimedOutException))

Operation [38] retried 10 times - error on calling get_indexed_slices for range offset 0000038 ((TimedOutException))

Operation [18] retried 10 times - error on calling get_indexed_slices for range offset 0000018 ((TimedOutException))

Operation [31] retried 10 times - error on calling get_indexed_slices for range offset 0000031 ((TimedOutException))

Operation [4] retried 10 times - error on calling get_indexed_slices for range offset 0000004 ((TimedOutException))

Operation [22] retried 10 times - error on calling get_indexed_slices for range offset 0000022 ((TimedOutException))

Operation [49] retried 10 times - error on calling get_indexed_slices for range offset 0000049 ((TimedOutException))

Operation [41] retried 10 times - error on calling get_indexed_slices for range offset 0000041 ((TimedOutException))

Operation [37] retried 10 times - error on calling get_indexed_slices for range offset 0000037 ((TimedOutException))

Operation [13] retried 10 times - error on calling get_indexed_slices for range offset 0000013 ((TimedOutException))

Operation [39] retried 10 times - error on calling get_indexed_slices for range offset 0000039 ((TimedOutException))

Operation [21] retried 10 times - error on calling get_indexed_slices for range offset 0000021 ((TimedOutException))

Operation [10] retried 10 times - error on calling get_indexed_slices for range offset 0000010 ((TimedOutException))

Operation [17] retried 10 times - error on calling get_indexed_slices for range offset 0000017 ((TimedOutException))

Operation [1] retried 10 times - error on calling get_indexed_slices for range offset 0000001 ((TimedOutException))

Operation [42] retried 10 times - error on calling get_indexed_slices for range offset 0000042 ((TimedOutException))

0,0,0,NaN,106
END


On Dec 2, 2011, at 2:45 PM, Janne Jalkanen wrote:

> 
> Would be glad to be of any help; it's kind of annoying.
> 
> * Nothing unusual on any nodes that I can see
> * Cannot reproduce on a single-node cluster; I see it only on our prod cluster which was running 0.6.13 until this point (cluster conf is attached to the JIRA issue mentioned below).
> 
> Let me know of anything that I can try, short of taking my production cluster offline :-P
> 
> /Janne
> 
> On Dec 2, 2011, at 20:42 , Jonathan Ellis wrote:
> 
>> The first step towards determining how serious it is, is showing us
>> how to reproduce it or otherwise narrowing down what could be causing
>> it, because timeouts can be caused by a lot of non-bug scenarios.
>> Does it occur for every query or just some?  Is there anything unusual
>> on the coordinator or replica nodes, like high CPU?  Can you reproduce
>> with the stress tool?  Can you reproduce on a single-node-cluster?
>> That kind of thing.
>> 
>> On Fri, Dec 2, 2011 at 12:18 PM, Pierre Belanger
>> <pierre.belanger@xobni.com> wrote:
>>> Hello,
>>> 
>>> Is this bug serious enough for 1.0.6 to come out shortly or not?
>>> 
>>> Thank you,
>>> PBR
>>> 
>>> 
>>> 
>>> On Thu, Dec 1, 2011 at 6:05 PM, Zhong Li <zli@voxeo.com> wrote:
>>>> 
>>>> After upgrade to 1.0.5 RangeSlice got timeout. Ticket
>>>> https://issues.apache.org/jira/browse/CASSANDRA-3551
>>>> 
>>>> On Dec 1, 2011, at 5:43 PM, Evgeniy Ryabitskiy wrote:
>>>> 
>>>>> +1
>>>>> After upgrade to 1.0.5 also have Timeout exception on Secondary Index
>>>>> search (get_indexed_slices API) .
>>>> 
>>> 
>> 
>> 
>> 
>> -- 
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
> 


This email was sent to you by Thomson Reuters, the global news and information company. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters.