Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=content-type
	:mime-version:subject:from:in-reply-to:date
	:content-transfer-encoding:message-id:references:to; q=dns; s=
	thelastpickle.com; b=VMA3XcvtGrF3WSftriYxGfCQdoUIw+xGC/0YY5AXk+V
	Q4zO6WqIJRj3ojv1ZOV+iEW/UgBxEDOty7v2cAdJxAl2EhQ+0Owutl/OrhZZ4hv2
	BcHlPbB6aSv1ffynUW/k9p7yFx7RkLqlNYKAJ96xh8VwzOiajcuUThzGc7g7rBIM
	=
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Apple Message framework v1244.3)
Subject: Re: Avoid Simultaneous Minor Compactions?
From: aaron morton <aaron@thelastpickle.com>
In-Reply-To: <A8060DD1-4303-4853-9D04-6CF8561B2E61@rhapsody.com>
Date: Tue, 23 Aug 2011 19:02:10 +1200
Content-Transfer-Encoding: quoted-printable
Message-Id: <35D8CC1B-2FFE-487F-A279-9563263C9AEF@thelastpickle.com>
References: <E93AB623-CDB4-4E8B-A4D0-1F5A58A5B690@rhapsody.com>
 <CANpsMSHqmupnvjSz5mnVbp+7YbF866-=t--aFL6F+ANkd6P+Og@mail.gmail.com>
 <CALdd-zjtwyRHsV4yLgcb_JMY54Be4GpSu7KTC0v1cjqFNyCGkw@mail.gmail.com>
 <A8060DD1-4303-4853-9D04-6CF8561B2E61@rhapsody.com>
To: user@cassandra.apache.org

Change one thing at a time and work out what metric it is you want to =
improve.=20

I would start with reducing compaction_throughput_mb_per_sec. Have a =
look in your logs for the "Enqueuing flush of Memtable=85" messages, =
count up how many serialised bytes you are flushing and then check it =
against the advice in the yaml file.=20

Cheers


-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 23/08/2011, at 11:36 AM, Hefeng Yuan wrote:

> Shall I lower this or increase it? Or probably ask in this way, do we =
suggest to let it run longer while using less CPU, or should we let it =
finish faster with more CPU usage?
> The problem we're facing is, with the default setting, they run slow =
and also eat a lot of CPU in the meanwhile.
>=20
> I'm thinking about the following changes, does this make sense?
> 1. lower the compaction thread priority
> 2. shorten the compaction threshold to 2~20
> 3. lower compaction_throughput_mb_per_sec to 10
>=20
> Thanks,
> Hefeng
>=20
> On Aug 22, 2011, at 8:09 AM, Jonathan Ellis wrote:
>=20
>> Specifically, look at compaction_throughput_mb_per_sec in =
cassandra.yaml
>>=20
>> On Mon, Aug 22, 2011 at 12:39 AM, Ryan King <ryan@twitter.com> wrote:
>>> You should throttle your compactions to a sustainable level.
>>>=20
>>> -ryan
>>>=20
>>> On Sun, Aug 21, 2011 at 10:22 PM, Hefeng Yuan <hfyuan@rhapsody.com> =
wrote:
>>>> We just noticed that at one time, 4 nodes were doing minor =
compaction together, each of them took 20~60 minutes.
>>>> We're on 0.8.1, 6 nodes, RF5.
>>>> This simultaneous compactions slowed down the whole cluster, we =
have local_quorum consistency level, therefore, dynamic_snitch is not =
helping us.
>>>>=20
>>>> Aside from lower down the compaction thread priority, is there any =
other way to tell the cluster hold on doing this if other nodes are =
already compacting?
>>>>=20
>>>> Thanks,
>>>> Hefeng
>>>=20
>>=20
>>=20
>>=20
>> --=20
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>=20