Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4B7D710148 for ; Wed, 2 Oct 2013 14:30:39 +0000 (UTC) Received: (qmail 17627 invoked by uid 500); 2 Oct 2013 14:30:36 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 15081 invoked by uid 500); 2 Oct 2013 14:30:28 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 14541 invoked by uid 99); 2 Oct 2013 14:30:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Oct 2013 14:30:27 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of cayiroglu@gmail.com designates 74.125.82.178 as permitted sender) Received: from [74.125.82.178] (HELO mail-we0-f178.google.com) (74.125.82.178) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Oct 2013 14:30:20 +0000 Received: by mail-we0-f178.google.com with SMTP id q59so852164wes.23 for ; Wed, 02 Oct 2013 07:30:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=XcYRM6KsMokvYHf2M0mCdUzHbCo/oi48DOEO4zdeoz0=; b=D1Ipst7jM16rSmZP+uNjtqeJGsxaFwW1bF9rzogk6ZxmEWZmCL7UVdJz2Ecxs1rMlS ebyvSVmLQiZeUxkvjsOu0g1yLeRodrZqeT8Anl2GYPNQs2Z2uHBWFQ0x3FU1nVy+tXsh GPZur90jSX2rTWdT+x5KsnUNqbjW7gW/2G0Z6yUbXN4VkbLqHTjIGAB6KxFij8Cc8e2K RaXw+1E7vKXv0b1dNDDtsaj+Keftf0qgu42e+i3ahVgyMAVO1V9SrYDQ5EJWCYCaZDdM V6AUscLwdEz7WMC2LJISy8bLDjVxoJ1z4liXdg7h9NcUSV3xYj9BlY2JuDBIrPRGpkwl NNMw== MIME-Version: 1.0 X-Received: by 10.180.182.15 with SMTP id ea15mr23715133wic.16.1380724200507; Wed, 02 Oct 2013 07:30:00 -0700 (PDT) Received: by 10.194.124.50 with HTTP; Wed, 2 Oct 2013 07:30:00 -0700 (PDT) In-Reply-To: References: Date: Wed, 2 Oct 2013 16:30:00 +0200 Message-ID: Subject: Re: Cassandra Heap Size for data more than 1 TB From: cem To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=047d7b62535a810f2e04e7c2e749 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b62535a810f2e04e7c2e749 Content-Type: text/plain; charset=ISO-8859-1 Have a look to index_interval. Cem. On Wed, Oct 2, 2013 at 2:25 PM, srmore wrote: > The version of Cassandra I am using is 1.0.11, we are migrating to 1.2.X > though. We had tuned bloom filters (0.1) and AFAIK making it lower than > this won't matter. > > Thanks ! > > > On Tue, Oct 1, 2013 at 11:54 PM, Mohit Anchlia wrote: > >> Which Cassandra version are you on? Essentially heap size is function of >> number of keys/metadata. In Cassandra 1.2 lot of the metadata like bloom >> filters were moved off heap. >> >> >> On Tue, Oct 1, 2013 at 9:34 PM, srmore wrote: >> >>> Does anyone know what would roughly be the heap size for cassandra with >>> 1TB of data ? We started with about 200 G and now on one of the nodes we >>> are already on 1 TB. We were using 8G of heap and that served us well up >>> until we reached 700 G where we started seeing failures and nodes flipping. >>> >>> With 1 TB of data the node refuses to come back due to lack of memory. >>> needless to say repairs and compactions takes a lot of time. We upped the >>> heap from 8 G to 12 G and suddenly everything started moving rapidly i.e. >>> the repair tasks and the compaction tasks. But soon (in about 9-10 hrs) we >>> started seeing the same symptoms as we were seeing with 8 G. >>> >>> So my question is how do I determine what is the optimal size of heap >>> for data around 1 TB ? >>> >>> Following are some of my JVM settings >>> >>> -Xms8G >>> -Xmx8G >>> -Xmn800m >>> -XX:NewSize=1200M >>> XX:MaxTenuringThreshold=2 >>> -XX:SurvivorRatio=4 >>> >>> Thanks ! >>> >> >> > --047d7b62535a810f2e04e7c2e749 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Have a look to index_interval.

Cem.


On We= d, Oct 2, 2013 at 2:25 PM, srmore <comomore@gmail.com> wrot= e:
The version of Cassand= ra I am using is 1.0.11, we are migrating to 1.2.X though. We had tuned blo= om filters (0.1) and AFAIK making it lower than this won't matter.

Thanks !


On Tue, Oct 1, 2013 at 11:54 PM, Mohit A= nchlia <mohitanchlia@gmail.com> wrote:
Which Cassandra version are you on? Essentially heap size is function of nu= mber of keys/metadata. In Cassandra 1.2 lot of the metadata like bloom filt= ers were moved off heap.


On Tue, Oct 1, 2013 at 9:34 PM, srmore <comomore@gmail.com>= wrote:
Does anyone know what= would roughly be the heap size for cassandra with 1TB of data ? We started= with about 200 G and now on one of the nodes we are already on 1 TB. We we= re using 8G of heap and that served us well up until we reached 700 G where= we started seeing failures and nodes flipping.

With 1 TB of data the node refuses to come back due to lack of me= mory. needless to say repairs and compactions takes a lot of time. We upped= the heap from 8 G to 12 G and suddenly everything started moving rapidly i= .e. the repair tasks and the compaction tasks. But soon (in about 9-10 hrs)= we started seeing the same symptoms as we were seeing with 8 G.

So my question is how do I determine what is the optimal size of = heap for data around 1 TB ?

Following are some of my JVM sett= ings

-Xms8G
-Xmx8G
-Xmn800m
-XX:NewSize=3D1200M
XX:MaxT= enuringThreshold=3D2
-XX:SurvivorRatio=3D4

Thanks !



--047d7b62535a810f2e04e7c2e749--