incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher J. Bottaro" <cjbott...@academicworks.com>
Subject Re: nodetool repair seems to increase linearly with number of keyspaces
Date Tue, 26 Nov 2013 18:17:43 GMT
We only have a single CF per keyspace.  Actually we have 2, but one is tiny
(only has 2 rows in it and is queried once a month or less).

Yup, using vnodes with 256 tokens.

Cassandra 1.2.10.

-- C


On Mon, Nov 25, 2013 at 2:28 PM, John Pyeatt <john.pyeatt@singlewire.com>wrote:

> Mr. Bottaro,
>
> About how many column families are in your keyspaces? We have 28 per
> keyspace.
>
> Are you using Vnodes? We are and they are set to 256
>
> What version of cassandra are you running. We are running 1.2.9
>
>
> On Mon, Nov 25, 2013 at 11:36 AM, Christopher J. Bottaro <
> cjbottaro@academicworks.com> wrote:
>
>> We have the same setup:  one keyspace per client, and currently about 300
>> keyspaces.  nodetool repair takes a long time, 4 hours with -pr on a single
>> node.  We have a 4 node cluster with about 10 gb per node.  Unfortunately,
>> we haven't been keeping track of the running time as keyspaces, or load,
>> increases.
>>
>> -- C
>>
>>
>> On Wed, Nov 20, 2013 at 6:53 AM, John Pyeatt <john.pyeatt@singlewire.com>wrote:
>>
>>> We have an application that has been designed to use potentially 100s of
>>> keyspaces (one for each company).
>>>
>>> One thing we are noticing is that nodetool repair across all of the
>>> keyspaces seems to increase linearly based on the number of keyspaces. For
>>> example, if we have a 6 node ec2 (m1.large) cluster across 3 Availability
>>> Zones and create 20 keyspaces a nodetool repair -pr on one node takes 3
>>> hours even with no data in any of the keyspaces. If I bump that up to 40
>>> keyspaces it takes 6 hours.
>>>
>>> Is this the behaviour you would expect?
>>>
>>> Is there anything you can think of (short of redesigning the cluster to
>>> limit keyspaces) to increase the performance of the nodetool repairs?
>>>
>>> My obvious concern is that as this application grows and we get more
>>> companies using our it we will eventually have too many keyspaces to
>>> perform repairs on the cluster.
>>>
>>> --
>>> John Pyeatt
>>> Singlewire Software, LLC
>>> www.singlewire.com
>>> ------------------
>>> 608.661.1184
>>> john.pyeatt@singlewire.com
>>>
>>
>>
>
>
> --
> John Pyeatt
> Singlewire Software, LLC
> www.singlewire.com
> ------------------
> 608.661.1184
> john.pyeatt@singlewire.com
>

Mime
View raw message