Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1E73F9761 for ; Wed, 16 May 2012 12:36:38 +0000 (UTC) Received: (qmail 53327 invoked by uid 500); 16 May 2012 12:36:35 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 53250 invoked by uid 500); 16 May 2012 12:36:35 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 53240 invoked by uid 99); 16 May 2012 12:36:35 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 May 2012 12:36:35 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yiming.sun@gmail.com designates 74.125.82.44 as permitted sender) Received: from [74.125.82.44] (HELO mail-wg0-f44.google.com) (74.125.82.44) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 May 2012 12:36:30 +0000 Received: by wgbdr13 with SMTP id dr13so486343wgb.25 for ; Wed, 16 May 2012 05:36:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=ehXIz73vcPiioQkUUhm6x/ijYGn1UW2HIdo1kkjdZiY=; b=XwFRHC8D+4mglQ2rspcM4mFgBOrumUrV8HSK4UCDlWaeEDfaVO8nY86G5SD9lQGwle FyjX7XYV+sYXBmamk/PXqPZXcWE7BMzJujipfUnKs+gpk4duFQGF9w92EzL6o2/CxOFM 5HaRn7Kj/8ASC7D8nW6FHc61ncKSZOjcrvenQK5WQtHbpV/ZA4HFRKZbzbfjokkjy5WU 0PHydCBzO6+8grlibEsPlJiFvEhBTg9IxAFWYNSOJ5rBdf+LZY7sy+KvnnyiwUORn3Pr Zgujbc2VCwYkB7hJcypUF6NEepbRgHHmlN6SIc9yBZ4uioTgOGnNrN8viy+8fUn5xjpO OTIw== Received: by 10.180.104.137 with SMTP id ge9mr7548683wib.20.1337171769397; Wed, 16 May 2012 05:36:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.232.92 with HTTP; Wed, 16 May 2012 05:35:49 -0700 (PDT) In-Reply-To: <8FADD602-74E1-475C-A323-6FB68F41B7AB@thelastpickle.com> References: <8FADD602-74E1-475C-A323-6FB68F41B7AB@thelastpickle.com> From: Yiming Sun Date: Wed, 16 May 2012 08:35:49 -0400 Message-ID: Subject: Re: need some clarification on recommended memory size To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=f46d0418263851933204c02690ff X-Virus-Checked: Checked by ClamAV on apache.org --f46d0418263851933204c02690ff Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Thanks Aaron. The reason I raised the question about memory requirements is because we are seeing some very low performance on cassandra read. We are using cassandra as the backend for an IR repository, and granted the size of each column is very small (OCRed text). Each row represents a book volume, and the columns of the row represent pages of the volume. The average size of a column text is 2-3KB, and each row has about 250 columns (varies quite a bit from one volume to another). The read rate that I have been seeing is about 3MB/sec, and that is reading the raw bytes... using string serializer the rate is even lower, about 2.2MB/sec. To retrieve each volume, a slice query is used via Hector that specifies the row key (the volume), and a list of column keys (pages), and the consistency level is set to ONE. So I am a bit lost in trying to figure out how to increase the performance. Using JNA may help, but a blog article seems to say it only increase 13%, which is not very significant when the base performance is in single-digit MBs. Do you have any suggestions? Oh, another thing is you mentioned memory mapped files. Our environment is virtualized, and the disks are actually SAN through fiber channels, so I don't know if that has impact on performance as well. Would greatly appreciate any help. Thanks. -- Y. On Wed, May 16, 2012 at 5:48 AM, aaron morton wrot= e: > The JVM will not swap out if you have JNA.jar in the path or you have > disabled swap on the machine (the simplest thing to do). > > Cassandra uses memory mapped file access. If you have 16GB of ram, 8 will > go to the JVM and the rest can be used by the os to cache files. (Plus th= e > off heap stuff) > > Cheers > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 16/05/2012, at 11:12 AM, Yiming Sun wrote: > > Thanks Tyler... so my understanding is, even if Cassandra doesn't do > off-heap caching, by having a large-enough memory, it minimize the chance > of swapping the java heap to a disk. Is that correct? > > -- Y. > > On Tue, May 15, 2012 at 6:26 PM, Tyler Hobbs wrote: > >> On Tue, May 15, 2012 at 3:19 PM, Yiming Sun wrote= : >> >>> Hello, >>> >>> I was reading the Apache Cassandra 1.0 Documentation PDF dated May 10, >>> 2012, and had some questions on what the recommended memory size is. >>> >>> Below is the snippet from the PDF. Bullet 1 suggests to have 16-32GB o= f >>> RAM, yet Bullet 2 suggests to limit Java heap size to no more than 8GB.= My >>> understanding is that Cassandra is implemented purely in Java, so all >>> memory it sees and uses is the JVM Heap. >>> >> >> The main way that additional RAM helps is through the OS page cache, >> which will store hot portions of SSTables in memory. Additionally, >> Cassandra can now do off-heap caching. >> >> >> >>> So can someone help me understand the discrepancy between 16-32GB of >>> RAM and 8GB of heap? Thanks. >>> >>> =3D=3D snippet =3D=3D >>> Memory >>> The more memory a Cassandra node has, the better read performance. More >>> RAM allows for larger cache sizes and >>> reduces disk I/O for reads. More RAM also allows memory tables >>> (memtables) to hold more recently written data. Larger >>> memtables lead to a fewer number of SSTables being flushed to disk and >>> fewer files to scan during a read. The ideal >>> amount of RAM depends on the anticipated size of your hot data. >>> >>> =95 For dedicated hardware, a minimum of than 8GB of RAM is needed. >>> DataStax recommends 16GB - 32GB. >>> >>> =95 Java heap space should be set to a maximum of 8GB or half of your >>> total RAM, whichever is lower. (A greater >>> heap size has more intense garbage collection periods.) >>> >>> =95 For a virtual environment use a minimum of 4GB, such as Amazon EC2 >>> Large instances. For production clusters >>> with a healthy amount of traffic, 8GB is more common. >>> >> >> >> >> -- >> Tyler Hobbs >> DataStax >> >> > > --f46d0418263851933204c02690ff Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Thanks Aaron. =A0The reason I raised the question about memory requirements= is because we are seeing some very low performance on cassandra read.
=
We are using cassandra as the backend for an IR repository, = and granted the size of each column is very small (OCRed text). =A0Each row= represents a book volume, and the columns of the row represent pages of th= e volume. =A0The average size of a column text is 2-3KB, and each row has a= bout 250 columns (varies quite a bit from one volume to another).

The read rate that I have been seeing is about 3MB/sec,= and that is reading the raw bytes... using string serializer the rate is e= ven lower, about 2.2MB/sec. =A0 To retrieve each volume, a slice query is u= sed via Hector that specifies the row key (the volume), and a list of colum= n keys (pages), and the consistency level is set to ONE. =A0So I am a bit l= ost in trying to figure out how to increase the performance. =A0Using JNA m= ay help, but a blog article seems to say it only increase 13%, which is not= very significant when the base performance is in single-digit MBs.

Do you have any suggestions?

O= h, another thing is you mentioned memory mapped files. =A0Our environment i= s virtualized, and the disks are actually SAN through fiber channels, so I = don't know if that has impact on performance as well. =A0Would greatly = appreciate any help. =A0Thanks.

-- Y.

On Wed, = May 16, 2012 at 5:48 AM, aaron morton <aaron@thelastpickle.com&g= t; wrote:
The JVM = will not swap out if you have JNA.jar in the path or you have disabled swap= on the machine (the simplest thing to do).=A0

Cassandra uses memory mapped file access. If you have 16GB o= f ram, 8 will go to the JVM and the rest can be used by the os to cache fil= es. (Plus the off heap stuff)

Cheers
=A0
<= div style=3D"word-wrap:break-word">
-----------------
Aaron Morton
Freelance Deve= loper
@aaronmorton

On 16/05/2012, at 11:12 AM, Yiming Sun wrote:

Thanks Tyler... so my understanding is, even if Cassand= ra doesn't do off-heap caching, by having a large-enough memory, it min= imize the chance of swapping the java heap to a disk. =A0Is that correct?
-- Y.

On Tue, May 15, 2012 at 6:26 PM, Tyler Hobbs= <tyler@datastax.com> wrote:
On Tue, May 15, 2012 at 3:19 PM, Yiming Sun= <yiming.sun@gmail.com> wrote:
Hello,

I was reading the Apache Cassandra 1.0 Documentat= ion PDF dated May 10, 2012, and had some questions on what the recommended = memory size is.

Below is the snippet from the PDF.= =A0Bullet 1 suggests to have 16-32GB of RAM, yet Bullet 2 suggests to limi= t Java heap size to no more than 8GB. =A0My understanding is that Cassandra= is implemented purely in Java, so all memory it sees and uses is the JVM H= eap.

The main way that additional RAM helps is throu= gh the OS page cache, which will store hot portions of SSTables in memory. = Additionally, Cassandra can now do off-heap caching.

=A0
=A0So can someone help me understand the discrepancy between 16-32GB = of RAM and 8GB of heap? =A0Thanks.

=3D=3D snippet =3D=3D
Memory
T= he more memory a Cassandra node has, the better read performance. More RAM = allows for larger cache sizes and
reduces disk I/O for reads. Mor= e RAM also allows memory tables (memtables) to hold more recently written d= ata. Larger
memtables lead to a fewer number of SSTables being flushed to disk and= fewer files to scan during a read. The ideal
amount of RAM depen= ds on the anticipated size of your hot data.

=95 For dedicated hardware, a minimum of than 8GB of RAM is needed. DataSta= x recommends 16GB - 32GB.

=95 Java heap space should be set to a maximum of 8GB o= r half of your total RAM, whichever is lower. (A greater
heap siz= e has more intense garbage collection periods.)

=95 For a virtual environment use a minimum of 4GB, such as Amazon EC2 Larg= e instances. For production clusters
with a healthy amount of tra= ffic, 8GB is more common.



--
Tyler Hobbs
DataStax
<= br>



--f46d0418263851933204c02690ff--