Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 97586 invoked from network); 21 Apr 2010 12:11:14 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 21 Apr 2010 12:11:14 -0000 Received: (qmail 39076 invoked by uid 500); 21 Apr 2010 12:11:13 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 39054 invoked by uid 500); 21 Apr 2010 12:11:13 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 39046 invoked by uid 99); 21 Apr 2010 12:11:12 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Apr 2010 12:11:12 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of nithril@gmail.com designates 209.85.218.222 as permitted sender) Received: from [209.85.218.222] (HELO mail-bw0-f222.google.com) (209.85.218.222) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Apr 2010 12:11:04 +0000 Received: by bwz22 with SMTP id 22so6987415bwz.5 for ; Wed, 21 Apr 2010 05:10:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:received:message-id :subject:from:to:content-type; bh=1d4IJibZfD58IYZCXiKFH99oYxb+QvR23a2C0fk9PU0=; b=FrQbqagmd9BBZrz3ARpZppeLPiJm3QhCUuKfgNHaFQVqTuI3CCZ/vpVf3ELiJlqh+j hmgSZ6m3gUTnsZ+5cGoWy5ijbA74WhxIIRL3iM7oO98fZ7BytQZMkMSZbdMTmLo0Fq4i wHQWhCuOswPjHg41cqBrNV1Bp+6FJzAAYih2A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=dfYrf7rDgpnXZRN5ZWz6ipd2nqTvVGCjsaIhPmK9X9Xn3K8S/i+ulLu58LvPgpn0jC zG9O0Vemf7paqCNnhNUpJvCsxv3fgK7oiUbnCBiBhQBFFCTcfB9CRklHEBFyPtdPUgv2 zEsXX+P0iVFSiqo4zSr2asRETr4JwV+2vkKjg= MIME-Version: 1.0 Received: by 10.204.99.143 with HTTP; Wed, 21 Apr 2010 05:10:44 -0700 (PDT) Date: Wed, 21 Apr 2010 14:10:44 +0200 Received: by 10.204.9.152 with SMTP id l24mr1886924bkl.193.1271851844143; Wed, 21 Apr 2010 05:10:44 -0700 (PDT) Message-ID: Subject: Cassandra tuning for running test on a desktop From: Nicolas Labrot To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0015175883c8608a070484be1561 X-Virus-Checked: Checked by ClamAV on apache.org --0015175883c8608a070484be1561 Content-Type: text/plain; charset=ISO-8859-1 Hello, For my first message I will first thanks Cassandra contributors for their great works. I have a parameter issue with Cassandra (I hope it's just a parameter issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a simple dual core with 4GB of RAM on WinXP. I have keep the default JVM option inside cassandra.bat (Xmx1G) I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF (named Super1). The insertion go to 1 millions of SC (without slowdown) and Cassandra crash because of an OOM. (I store an average of 100 bytes per SC with a max of 10kB). I have aggressively decreased all the memories parameters without any respect to the consistency (My config is here [1]), the cache is turn off but Cassandra still go to OOM. I have joined the last line of the Cassandra life [2]. What can I do to fix my issue ? Is there another solution than increasing the Xmx ? Thanks for your help, Nicolas [1] org.apache.cassandra.locator.RackUnawareStrategy 1 org.apache.cassandra.locator.EndPointSnitch 32 auto 64 64 16 4 64 16 32 0.01 0.01 60 4 8 [2] INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', position=5417524) INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755 INFO 13:36:41,062 Writing Memtable(Super1)@15385755 INFO 13:36:42,062 Completed flushing d:\cassandra\data\Keyspace1\Super1-711-Data.db INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', position=6065637) INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910 INFO 13:36:45,796 Writing Memtable(Super1)@15578910 INFO 13:36:46,109 Completed flushing d:\cassandra\data\Keyspace1\Super1-712-Data.db INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed leaving 922392600 used; max is 1174208512 INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', position=6722241) INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872 INFO 13:36:54,593 Writing Memtable(Super1)@24468872 INFO 13:36:55,421 Completed flushing d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError: Java heap space INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed leaving 971904520 used; max is 1174208512 --0015175883c8608a070484be1561 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello,

For my first message I will first thanks Cassandra contributo= rs for their great works.

I have a parameter issue with Cassandra (= I hope it's just a parameter issue). I'm using Cassandra 6.0.1 with= Hector client on my desktop. It's a simple dual core with 4GB of RAM o= n WinXP. I have keep the default JVM option inside cassandra.bat (Xmx1G)
I'm trying to insert 3 millions of SC with 6 Columns each inside 1 = CF (named Super1). The insertion go to 1 millions of SC (without slowdown) = and Cassandra crash because of an OOM. (I store an average of 100 bytes per= SC with a max of 10kB).
I have aggressively decreased all the memories parameters without any respe= ct to the consistency (My config is here [1]), the cache is turn off but Ca= ssandra still go to OOM. I have joined the last line of the Cassandra life = [2].

What can I do to fix my issue ?=A0 Is there another solution than incre= asing the Xmx ?

Thanks for your help,

Nicolas



=

[1]
=A0 <Keyspaces>
=A0=A0=A0 <Keyspace Name=3D"= ;Keyspace1">
=A0=A0=A0=A0=A0 <ColumnFamily Name=3D"Super1"
=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 ColumnType=3D"Super"= ;
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 CompareWith= =3D"BytesType"
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 CompareSubcolumnsWith=3D"BytesType" />
=A0=A0=A0=A0=A0 <ReplicaPlacementStrategy>org.apache.cassandra.locato= r.RackUnawareStrategy</ReplicaPlacementStrategy>
=A0=A0=A0=A0=A0 &= lt;ReplicationFactor>1</ReplicationFactor>
=A0=A0=A0=A0=A0 <= EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointS= nitch>
=A0=A0=A0 </Keyspace>
=A0 </Keyspaces>
=A0 <CommitLogR= otationThresholdInMB>32</CommitLogRotationThresholdInMB>

= =A0 <DiskAccessMode>auto</DiskAccessMode>
=A0 <RowWarning= ThresholdInMB>64</RowWarningThresholdInMB>
=A0 <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
=A0 <= FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
=A0 <Fl= ushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
=A0 <Col= umnIndexSizeInKB>64</ColumnIndexSizeInKB>

=A0 <MemtableThroughputInMB>16</MemtableThroughputInMB>
= =A0 <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB= >
=A0 <MemtableOperationsInMillions>0.01</MemtableOperations= InMillions>
=A0 <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMill= ions>
=A0 <MemtableFlushAfterMinutes>60</MemtableFlushAfterM= inutes>
=A0 <ConcurrentReads>4</ConcurrentReads>
=A0 &= lt;ConcurrentWrites>8</ConcurrentWrites>
</Storage>


[2]
=A0INFO 13:36:41,062 Super1 has reached = its threshold; switching in a fresh Memtable at CommitLogContext(file=3D= 9;d:/cassandra/commitlog\CommitLog-1271849783703.log', position=3D54175= 24)
=A0INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
=A0INF= O 13:36:41,062 Writing Memtable(Super1)@15385755
=A0INFO 13:36:42,062 Co= mpleted flushing d:\cassandra\data\Keyspace1\Super1-711-Data.db
=A0INFO = 13:36:45,781 Super1 has reached its threshold; switching in a fresh Memtabl= e at CommitLogContext(file=3D'd:/cassandra/commitlog\CommitLog-12718497= 83703.log', position=3D6065637)
=A0INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
=A0INF= O 13:36:45,796 Writing Memtable(Super1)@15578910
=A0INFO 13:36:46,109 Co= mpleted flushing d:\cassandra\data\Keyspace1\Super1-712-Data.db
=A0INFO = 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed leavin= g 922392600 used; max is 1174208512
=A0INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh= Memtable at CommitLogContext(file=3D'd:/cassandra/commitlog\CommitLog-= 1271849783703.log', position=3D6722241)
=A0INFO 13:36:54,593 Enqueui= ng flush of Memtable(Super1)@24468872
=A0INFO 13:36:54,593 Writing Memtable(Super1)@24468872
=A0INFO 13:36:55,= 421 Completed flushing d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.l= ang.OutOfMemoryError: Java heap space
=A0INFO 13:37:08,281 GC for Concur= rentMarkSweep: 5561 ms, 9432 reclaimed leaving 971904520 used; max is 11742= 08512

--0015175883c8608a070484be1561--