Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2846B9F6E for ; Mon, 27 Feb 2012 20:52:11 +0000 (UTC) Received: (qmail 45799 invoked by uid 500); 27 Feb 2012 20:52:08 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 45764 invoked by uid 500); 27 Feb 2012 20:52:08 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 45756 invoked by uid 99); 27 Feb 2012 20:52:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Feb 2012 20:52:08 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a51.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Feb 2012 20:52:01 +0000 Received: from homiemail-a51.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a51.g.dreamhost.com (Postfix) with ESMTP id 642E32E806D for ; Mon, 27 Feb 2012 12:51:05 -0800 (PST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; q=dns; s=thelastpickle.com; b=EBeuWchFXR /K6ffygxsbKkjQlk1sQ6bX9TMahtWTYrsbLNERsXD1flFt753mQGuxmY0XEBvL+k W5JUh4j7bQw+s0js5i8+WfXfCh7s59/l37lZdbNlJDwgCz6EwYkPRBzLUgwYmtks Yrxwp834WCQaL41I+itONROaOVfWSJ52s= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; s=thelastpickle.com; bh=llaivChUs0/OLiBz mgfWB1rv/5U=; b=eX5LsYt+tnqf0lt7yHKxHYc0flYXA6GYoMDw7dFT7Az5GLrX +hlzXCVoYFv7mSRMz189OURnk1Ur8rW5xcEvJZfBiUygmd4xZng4xsITvz8kt2nT JiB1+6rUMupj1Sy4hqnbohAqUacx6eQL5SLio4T0IO73kDLe636DTf9h/Rw= Received: from 202-126-206-217.vectorcommunications.net.nz (unknown [202.126.206.217]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a51.g.dreamhost.com (Postfix) with ESMTPSA id E82202E8057 for ; Mon, 27 Feb 2012 12:51:04 -0800 (PST) From: aaron morton Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: multipart/alternative; boundary="Apple-Mail=_ABF04D9B-4797-4840-9BD2-FA260F12DCAC" Subject: Re: Frequency of Flushing in 1.0 Date: Tue, 28 Feb 2012 09:51:03 +1300 In-Reply-To: To: user@cassandra.apache.org References: <4F4A4B3D.4070201@sendmail.cz> <64B4139C-2181-4712-935C-6F4A296E8F96@thelastpickle.com> Message-Id: X-Mailer: Apple Mail (2.1257) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_ABF04D9B-4797-4840-9BD2-FA260F12DCAC Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 yes, reducing commitlog_total_space_in_mb will reduce the amount of = space needed by the commit logs.=20 > memtable_total_space_in_mb controls how often sstables are flushed to disk, this does not really = affect the commit log. Other than the fact that a commit log segment = cannot be deleted until the changes in the sstable have been flushed. = But commitlog_total_space_in_mb is the correct way to control that.=20 Hope that helps.=20 ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 27/02/2012, at 4:48 PM, Xaero S wrote: >=20 > The challenge that we face is that our commitlog disk capacity is much = much less (under 10 GB in some cases) than the disk capacity of = SSTables. So we cannot really have the commitlog data continuously = growing. This is the reason that we need to be able to tune the the way = we flush the memtables. =46rom this link - = http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memor= y-and-disk-space-management , it looks like = "commitlog_total_space_in_mb" is the parameter to control the rate at = which memtables get flushed. Also it seems "memtable_total_space_in_mb" = is also another setting to play with. > we are planning to do some load testing with changes to these two = settings, but can anyone confirm that i am headed in the right = direction? Or any other pointers on this? >=20 >=20 > On Sun, Feb 26, 2012 at 5:26 PM, Mohit Anchlia = wrote: >=20 >=20 > On Sun, Feb 26, 2012 at 12:18 PM, aaron morton = wrote: > Nathan Milford has a post about taking a node down=20 >=20 > http://blog.milford.io/2011/11/rolling-upgrades-for-cassandra/ >=20 > The only thing I would do differently would be turn off thrift first. >=20 > Cheers > =20 > Isn't decomission meant to do the same thing as disablethrift and = gossip? >=20 >=20 > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com >=20 > On 27/02/2012, at 4:35 AM, Edward Capriolo wrote: >=20 >> If you are doing a planned maintenance you can flush first as well >> ensuring the that the commit logs will not be as large. >>=20 >> On Sun, Feb 26, 2012 at 10:09 AM, Radim Kolar = wrote: >>>> if a node goes down, it will take longer for commitlog replay. >>>=20 >>> commit log replay time is insignificant. most time during node = startup is >>> wasted on index sampling. Index sampling here runs for about 15 = minutes. >=20 >=20 >=20 --Apple-Mail=_ABF04D9B-4797-4840-9BD2-FA260F12DCAC Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 yes, = reducing commitlog_total_space_in_mb will reduce the amount = of space needed by the commit logs. 

memtable_total_space_in_mb
controls how = often sstables are flushed to disk, this does not really affect the = commit log. Other than the fact that a commit log segment cannot be = deleted until the changes in the sstable have been flushed. = But commitlog_total_space_in_mb is the correct way to control = that. 

Hope that = helps. 

=
http://www.thelastpickle.com

On 27/02/2012, at 4:48 PM, Xaero S wrote:


The challenge that we face is = that our commitlog disk capacity is much much less (under 10 GB in some = cases) than the disk capacity of SSTables. So we cannot really have the = commitlog data continuously growing. This is the reason that we need to = be able to tune the the way we flush the memtables. =46rom this link - = http://www.datastax.com/dev/blog/what= s-new-in-cassandra-1-0-improved-memory-and-disk-space-management , = it looks like "commitlog_total_space_in_mb" is the parameter to control = the rate at which memtables get flushed. Also it seems = "memtable_total_space_in_mb" is also another setting to play = with.
we are planning to do some = load testing with changes to these two settings, but can anyone confirm = that i am headed in the right direction? Or any other pointers on = this?


On Sun, Feb 26, 2012 at 5:26 PM, = Mohit Anchlia <mohitanchlia@gmail.com> wrote:


On Sun, Feb 26, 2012 at = 12:18 PM, aaron morton <aaron@thelastpickle.com> wrote:
Nathan Milford has a post about = taking a node down =20


The only thing I would do differently would be turn off thrift = first.

Cheers
 
Isn't decomission meant to do the same thing as disablethrift = and gossip?


-----------------
Aaron Morton
Freelance Developer
@aaronmorton

On 27/02/2012, at 4:35 AM, Edward Capriolo wrote:

If you are doing a planned maintenance you can flush first as = well
ensuring the that the commit logs will not be as = large.

On Sun, Feb 26, 2012 at 10:09 AM, Radim Kolar <hsn@sendmail.cz> = wrote:
if a node goes down, it will take longer for = commitlog replay.

commit log replay time is insignificant. most = time during node startup is
wasted on index sampling. Index sampling here = runs for about 15 = minutes.




= --Apple-Mail=_ABF04D9B-4797-4840-9BD2-FA260F12DCAC--