Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 94774 invoked from network); 6 Apr 2011 20:30:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 6 Apr 2011 20:30:08 -0000 Received: (qmail 67013 invoked by uid 500); 6 Apr 2011 20:30:06 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 66986 invoked by uid 500); 6 Apr 2011 20:30:06 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 66978 invoked by uid 99); 6 Apr 2011 20:30:06 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Apr 2011 20:30:06 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of paulchoi@plaxo.com designates 64.160.187.196 as permitted sender) Received: from [64.160.187.196] (HELO plaxo.com) (64.160.187.196) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Apr 2011 20:30:02 +0000 Received: from exch2.plaxo.com (exch2.plaxo.com [172.20.0.56]) by corpmta2.plaxo.com (Postfix) with ESMTP id E06B217AF for ; Wed, 6 Apr 2011 13:29:41 -0700 (PDT) Received: from exch1.plaxo.com ([169.254.1.85]) by exch2.plaxo.com ([fe80::c50f:efd9:593d:81c5%10]) with mapi id 14.01.0270.001; Wed, 6 Apr 2011 13:29:41 -0700 From: Paul Choi To: "user@cassandra.apache.org" Subject: Re: Abnormal memory consumption Thread-Topic: Abnormal memory consumption Thread-Index: AQHL8HnpPxjh+gpNZUm5W36XygVMtJROEDEAgAAoioCAACxwAIAABgOAgAAO7oCAAArtAIAAPXeAgAJpuQA= Date: Wed, 6 Apr 2011 20:29:40 +0000 Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.2.0.101115 x-originating-ip: [172.20.1.186] Content-Type: multipart/alternative; boundary="_000_C9C1F99811C7Apaulchoiplaxocom_" MIME-Version: 1.0 --_000_C9C1F99811C7Apaulchoiplaxocom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable You can use JMX over ssh by doing this: http://blog.reactive.org/2011/02/connecting-to-cassandra-jmx-via-ssh.html Basically, you use SSH -D to do dynamic application port forwarding. In terms of scaling, you'll be able to afford 120GB RAM/node in 3 years if = you're successful. Or, a machine with much less RAM and flash-based storage= . :) Seriously, though, the formula in the tuning guidelines is a guideline. You= can probably get acceptable performance with much less. If not, you can sh= ard your app such that you host a few Cfs per cluster. I doubt you'll need = to though. From: openvictor Open > Reply-To: > Date: Mon, 4 Apr 2011 18:24:25 -0400 To: > Subject: Re: Abnormal memory consumption Okay, I see. But isn't there a big issue for scaling here ? Imagine that I am the developper of a certain very successful website : At = year 1 I need 20 CF. I might need to have 8Gb of RAM. Year 2 I need 50 CF b= ecause I added functionalities to my wonderful webiste will I need 20 Gb of= RAM ? And if at year three I had 300 Column families, will I need 120 Gb o= f ram / node ? Or did I miss something about memory consuption ? Thank you very much, Victor 2011/4/4 Peter Schuller > > And about the production 7Gb or RAM is sufficient ? Or 11 Gb is the minim= um > ? > Thank you for your inputs for the JVM I'll try to tune that Production mem reqs are mostly dependent on memtable thresholds: http://www.datastax.com/docs/0.7/operations/tuning If you enable key caching or row caching, you will have to adjust accordingly as well. -- / Peter Schuller --_000_C9C1F99811C7Apaulchoiplaxocom_ Content-Type: text/html; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable
You can use JMX over ssh by doing this:
http://blog.reactive.org/2011/02/connecting-to-cassandra-jmx-via-ssh.h= tml
Basically, you use SSH -D to do dynamic application port forwarding.

In terms of scaling, you'll be able to afford 120GB RAM/node in 3 year= s if you're successful. Or, a machine with much less RAM and flash-based st= orage. :)
Seriously, though, the formula in the tuning guidelines is a guideline= . You can probably get acceptable performance with much less. If not, you c= an shard your app such that you host a few Cfs per cluster. I doubt you'll = need to though.


From: openvictor Open <openvictor@gmail.com>
Reply-To: <user@cassandra.apache.org>
Date: Mon, 4 Apr 2011 18:24:25 -040= 0
To: <user@cassandra.apache.org>
Subject: Re: Abnormal memory consum= ption

Okay, I see. But isn't there a big issue for scaling here ?
Imagine that I am the developper of a certain very successful website : At = year 1 I need 20 CF. I might need to have 8Gb of RAM. Year 2 I need 50 CF b= ecause I added functionalities to my wonderful webiste will I need 20 Gb of= RAM ? And if at year three I had 300 Column families, will I need 120 Gb of ram / node ? Or did I miss some= thing about memory consuption ?

Thank you very much,

Victor

2011/4/4 Peter Schuller <peter.schuller@infidyne.com>
> And about the production 7Gb or RAM is sufficient ? = Or 11 Gb is the minimum
> ?
> Thank you for your inputs for the JVM I'll try to tune that

Production mem reqs are mostly dependent on memtable thresholds:

  http://www.datastax.com/docs/0.7/operations/tuning

If you enable key caching or row caching, you will have to adjust
accordingly as well.

--
/ Peter Schuller

 --_000_C9C1F99811C7Apaulchoiplaxocom_--