Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 98651 invoked from network); 21 Apr 2010 12:15:04 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 21 Apr 2010 12:15:04 -0000 Received: (qmail 45672 invoked by uid 500); 21 Apr 2010 12:15:03 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 45649 invoked by uid 500); 21 Apr 2010 12:15:03 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 45641 invoked by uid 99); 21 Apr 2010 12:15:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Apr 2010 12:15:03 +0000 X-ASF-Spam-Status: No, hits=0.8 required=10.0 tests=AWL,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of greenemj@gmail.com designates 209.85.222.203 as permitted sender) Received: from [209.85.222.203] (HELO mail-pz0-f203.google.com) (209.85.222.203) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Apr 2010 12:14:57 +0000 Received: by pzk41 with SMTP id 41so5092540pzk.10 for ; Wed, 21 Apr 2010 05:14:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=elmL0YRRha6ei+rFKrROKCqxzltvR7k/ZrlhOScIVhM=; b=Fr6oWYdbSduT188aJ62sC4Dso+tLnYbB2C4PgXlWvORxVzx783paJgo7f/Z67Ri4lJ oZ48HfN6+aPs8AODy8ssd0V9v/7OViTZ3C0V3yeP4SM2BhQwQyaSN9qU8UEQoomcujSK K0T0kmweFSp3KPe2KaJFjX3Ht28dqV7G6d1PU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=YNZ2bC16+cI+C6/TM4lzo0gE05NCvDMNJ44sygfYY8oeFVoPZTyS8cKj8o1DXuSust lWKlYpoSab51cgeNyiQ//dqG3EfIeXwUBh8xxcu+e4kbwkg6FmPqcNS1ovwPdiRH4806 cljs/oNW0/cODByOl0TfRvW5+59XigLeub2k4= Received: by 10.141.188.24 with SMTP id q24mr778894rvp.0.1271852077119; Wed, 21 Apr 2010 05:14:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.171.20 with HTTP; Wed, 21 Apr 2010 05:14:17 -0700 (PDT) In-Reply-To: References: From: Mark Greene Date: Wed, 21 Apr 2010 08:14:17 -0400 Message-ID: Subject: Re: Cassandra tuning for running test on a desktop To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=000e0cd17ed4437b190484be238b --000e0cd17ed4437b190484be238b Content-Type: text/plain; charset=ISO-8859-1 Trying increasing Xmx. 1G is probably not enough for the amount of inserts you are doing. On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot wrote: > Hello, > > For my first message I will first thanks Cassandra contributors for their > great works. > > I have a parameter issue with Cassandra (I hope it's just a parameter > issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a > simple dual core with 4GB of RAM on WinXP. I have keep the default JVM > option inside cassandra.bat (Xmx1G) > > I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF > (named Super1). The insertion go to 1 millions of SC (without slowdown) and > Cassandra crash because of an OOM. (I store an average of 100 bytes per SC > with a max of 10kB). > I have aggressively decreased all the memories parameters without any > respect to the consistency (My config is here [1]), the cache is turn off > but Cassandra still go to OOM. I have joined the last line of the Cassandra > life [2]. > > What can I do to fix my issue ? Is there another solution than increasing > the Xmx ? > > Thanks for your help, > > Nicolas > > > > > > [1] > > > ColumnType="Super" > CompareWith="BytesType" > CompareSubcolumnsWith="BytesType" /> > > org.apache.cassandra.locator.RackUnawareStrategy > 1 > > org.apache.cassandra.locator.EndPointSnitch > > > 32 > > auto > 64 > 64 > 16 > 4 > 64 > > 16 > 32 > 0.01 > 0.01 > 60 > 4 > 8 > > > > [2] > INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh > Memtable at > CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', > position=5417524) > INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755 > INFO 13:36:41,062 Writing Memtable(Super1)@15385755 > INFO 13:36:42,062 Completed flushing > d:\cassandra\data\Keyspace1\Super1-711-Data.db > INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh > Memtable at > CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', > position=6065637) > INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910 > INFO 13:36:45,796 Writing Memtable(Super1)@15578910 > INFO 13:36:46,109 Completed flushing > d:\cassandra\data\Keyspace1\Super1-712-Data.db > INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed > leaving 922392600 used; max is 1174208512 > INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh > Memtable at > CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log', > position=6722241) > INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872 > INFO 13:36:54,593 Writing Memtable(Super1)@24468872 > INFO 13:36:55,421 Completed flushing > d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError: > Java heap space > INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed > leaving 971904520 used; max is 1174208512 > > --000e0cd17ed4437b190484be238b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Trying increasing Xmx. 1G is probably not enough for the amount of inserts = you are doing.

On Wed, Apr 21, 2010 at 8:= 10 AM, Nicolas Labrot <nithril@gmail.com> wrote:
Hello,

For my first message I will f= irst thanks Cassandra contributors for their great works.

I have a = parameter issue with Cassandra (I hope it's just a parameter issue). I&= #39;m using Cassandra 6.0.1 with Hector client on my desktop. It's a si= mple dual core with 4GB of RAM on WinXP. I have keep the default JVM option= inside cassandra.bat (Xmx1G)

I'm trying to insert 3 millions of SC with 6 Columns each inside 1 = CF (named Super1). The insertion go to 1 millions of SC (without slowdown) = and Cassandra crash because of an OOM. (I store an average of 100 bytes per= SC with a max of 10kB).
I have aggressively decreased all the memories parameters without any respe= ct to the consistency (My config is here [1]), the cache is turn off but Ca= ssandra still go to OOM. I have joined the last line of the Cassandra life = [2].

What can I do to fix my issue ?=A0 Is there another solution than incre= asing the Xmx ?

Thanks for your help,

Nicolas



=

[1]
=A0 <Keyspaces>
=A0=A0=A0 <Keyspace Name=3D"= ;Keyspace1">
=A0=A0=A0=A0=A0 <ColumnFamily Name=3D"Super1"
=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 ColumnType=3D"Super"= ;
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 CompareWith= =3D"BytesType"
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 CompareSubcolumnsWith=3D"BytesType" />
=A0=A0=A0=A0=A0 <ReplicaPlacementStrategy>org.apache.cassandra.locato= r.RackUnawareStrategy</ReplicaPlacementStrategy>
=A0=A0=A0=A0=A0 &= lt;ReplicationFactor>1</ReplicationFactor>
=A0=A0=A0=A0=A0 <= EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointS= nitch>
=A0=A0=A0 </Keyspace>
=A0 </Keyspaces>
=A0 <CommitLogR= otationThresholdInMB>32</CommitLogRotationThresholdInMB>

= =A0 <DiskAccessMode>auto</DiskAccessMode>
=A0 <RowWarning= ThresholdInMB>64</RowWarningThresholdInMB>
=A0 <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
=A0 <= FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
=A0 <Fl= ushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
=A0 <Col= umnIndexSizeInKB>64</ColumnIndexSizeInKB>

=A0 <MemtableThroughputInMB>16</MemtableThroughputInMB>
= =A0 <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB= >
=A0 <MemtableOperationsInMillions>0.01</MemtableOperations= InMillions>
=A0 <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMill= ions>
=A0 <MemtableFlushAfterMinutes>60</MemtableFlushAfterM= inutes>
=A0 <ConcurrentReads>4</ConcurrentReads>
=A0 &= lt;ConcurrentWrites>8</ConcurrentWrites>
</Storage>


[2]
=A0INFO 13:36:41,062 Super1 has reached = its threshold; switching in a fresh Memtable at CommitLogContext(file=3D= 9;d:/cassandra/commitlog\CommitLog-1271849783703.log', position=3D54175= 24)
=A0INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
=A0INF= O 13:36:41,062 Writing Memtable(Super1)@15385755
=A0INFO 13:36:42,062 Co= mpleted flushing d:\cassandra\data\Keyspace1\Super1-711-Data.db
=A0INFO = 13:36:45,781 Super1 has reached its threshold; switching in a fresh Memtabl= e at CommitLogContext(file=3D'd:/cassandra/commitlog\CommitLog-12718497= 83703.log', position=3D6065637)
=A0INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
=A0INF= O 13:36:45,796 Writing Memtable(Super1)@15578910
=A0INFO 13:36:46,109 Co= mpleted flushing d:\cassandra\data\Keyspace1\Super1-712-Data.db
=A0INFO = 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed leavin= g 922392600 used; max is 1174208512
=A0INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh= Memtable at CommitLogContext(file=3D'd:/cassandra/commitlog\CommitLog-= 1271849783703.log', position=3D6722241)
=A0INFO 13:36:54,593 Enqueui= ng flush of Memtable(Super1)@24468872
=A0INFO 13:36:54,593 Writing Memtable(Super1)@24468872
=A0INFO 13:36:55,= 421 Completed flushing d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.l= ang.OutOfMemoryError: Java heap space
=A0INFO 13:37:08,281 GC for Concur= rentMarkSweep: 5561 ms, 9432 reclaimed leaving 971904520 used; max is 11742= 08512


--000e0cd17ed4437b190484be238b--