Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 56DEE97E2 for ; Sun, 24 Jun 2012 18:05:26 +0000 (UTC) Received: (qmail 6654 invoked by uid 500); 24 Jun 2012 18:05:23 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 6626 invoked by uid 500); 24 Jun 2012 18:05:23 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 6598 invoked by uid 99); 24 Jun 2012 18:05:23 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 24 Jun 2012 18:05:23 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a47.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 24 Jun 2012 18:05:17 +0000 Received: from homiemail-a47.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTP id 35B94284058 for ; Sun, 24 Jun 2012 11:04:56 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; q=dns; s=thelastpickle.com; b=pZxCHMO7yI 0pi+il5G6eE+7354Ld648HFEx9AyKH/FF6AJ9O5qOriaYU+s9bQwjWxgX2meb2Hr lYBoIDaDR4rO9KZd7TYDJPbhF6psURNCFRoGAr2k6fhY3ae6kH3qPOgk3DFOOnN4 Q9LY7cMKOhoZlPZ9MWlzG82gohB3jUnIw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; s=thelastpickle.com; bh=1DTagKGV0vf4TsCW XeN8CoZSPEQ=; b=GSIldOoJGcBQ5YuWC7MY840s8QwYSGiu8nicFsqhaK1HU9KW 0oavZDrycKqfv2cTeB65VTcDxkBnjPq6VsqOosWez2t3Euax3ofB0vA3mg5sIwDl EyXfyQktoMku5GZnyeb0l/C+gOi/tTDG/DeDxr7RhtN2xXeBIWEkz947pCg= Received: from [172.16.1.4] (unknown [203.86.207.101]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTPSA id ABB63284057 for ; Sun, 24 Jun 2012 11:04:55 -0700 (PDT) From: aaron morton Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: multipart/alternative; boundary="Apple-Mail=_195A62E4-676E-41A5-A587-0B4A84A648D9" Subject: Re: Tiered compation on two disks Date: Mon, 25 Jun 2012 06:04:53 +1200 In-Reply-To: <4FE2FDC3.7000203@list-group.com> To: user@cassandra.apache.org References: <4FE2FDC3.7000203@list-group.com> Message-Id: <7ABADB07-6468-4874-AE59-3925A7419FD6@thelastpickle.com> X-Mailer: Apple Mail (2.1278) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_195A62E4-676E-41A5-A587-0B4A84A648D9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > I have a Cassandra installation where we plan to store 1Tb of data, = split between two 1Tb disks. In general it's a good idea to limit the per node storage to 300GB to = 400GB. This has more to do with operational issues that any particular = issue with cassandra. However storing a very large number of keys on a = single node can result in high memory usage while the server is idling, = and reduced read performance.=20 =20 > I know that tiered compaction needs 50% free disk space for worst case = situation.=20 Not really now days, but it's a good idea to treat 50% as a soft limit.=20= > How does this combine with the disk split?=20 Whenever a new file is written to disk it will use the data directory = with the most space. In general we recommend using a single data = directory.=20 Hope that helps.=20 ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 21/06/2012, at 10:56 PM, Flavio Baronti wrote: > Hi, >=20 > I have a Cassandra installation where we plan to store 1Tb of data, = split between two 1Tb disks. > Tiered compation should be better suited for our workload = (append-only, deletion of old data, few reads). > I know that tiered compaction needs 50% free disk space for worst case = situation. How does this combine with the disk split? What happens if I = have 500Gb of data in one disk and 500Gb in the other? Won't compaction = try to build a single 1Tb file, failing since there are only 500Gb free = on each disk? >=20 > Flavio >=20 --Apple-Mail=_195A62E4-676E-41A5-A587-0B4A84A648D9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
I have a Cassandra installation where = we plan to store 1Tb of data, split between two 1Tb = disks.
In general it's a good idea to limit the per = node storage to 300GB to 400GB. This has more to do with operational = issues that any particular issue with cassandra. However storing a very = large number of keys on a single node can result in high memory usage = while the server is idling, and reduced read = performance. 
 
I know = that tiered compaction needs 50% free disk space for worst case = situation. 
Not really now days, but it's a good = idea to treat 50% as a soft limit. 

How does this combine with the disk = split? 
Whenever a new file is written to disk it = will use the data directory with the most space. In general we recommend = using a single data directory. 

Hope that = helps. 


http://www.thelastpickle.com

On 21/06/2012, at 10:56 PM, Flavio Baronti wrote:

Hi,

I have a Cassandra installation where we = plan to store 1Tb of data, split between two 1Tb disks.
Tiered = compation should be better suited for our workload (append-only, = deletion of old data, few reads).
I know that tiered compaction needs = 50% free disk space for worst case situation. How does this combine with = the disk split? What happens if I have 500Gb of data in one disk and = 500Gb in the other? Won't compaction try to build a single 1Tb file, = failing since there are only 500Gb free on each = disk?

Flavio


= = --Apple-Mail=_195A62E4-676E-41A5-A587-0B4A84A648D9--