incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Jones <MJo...@imagehawk.com>
Subject RE: Cassandra tuning for running test on a desktop
Date Wed, 21 Apr 2010 12:54:45 GMT
On my 4GB machine I'm giving it 3GB and having no trouble with 60+ million 500 byte columns

From: Nicolas Labrot [mailto:nithril@gmail.com]
Sent: Wednesday, April 21, 2010 7:47 AM
To: user@cassandra.apache.org
Subject: Re: Cassandra tuning for running test on a desktop

I have try 1400M, and Cassandra OOM too.

Is there another solution ? My data isn't very big.

It seems that is the merge of the db

On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <greenemj@gmail.com<mailto:greenemj@gmail.com>>
wrote:
Trying increasing Xmx. 1G is probably not enough for the amount of inserts you are doing.

On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <nithril@gmail.com<mailto:nithril@gmail.com>>
wrote:
Hello,

For my first message I will first thanks Cassandra contributors for their great works.

I have a parameter issue with Cassandra (I hope it's just a parameter issue). I'm using Cassandra
6.0.1 with Hector client on my desktop. It's a simple dual core with 4GB of RAM on WinXP.
I have keep the default JVM option inside cassandra.bat (Xmx1G)

I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF (named Super1). The
insertion go to 1 millions of SC (without slowdown) and Cassandra crash because of an OOM.
(I store an average of 100 bytes per SC with a max of 10kB).
I have aggressively decreased all the memories parameters without any respect to the consistency
(My config is here [1]), the cache is turn off but Cassandra still go to OOM. I have joined
the last line of the Cassandra life [2].

What can I do to fix my issue ?  Is there another solution than increasing the Xmx ?

Thanks for your help,

Nicolas





[1]
  <Keyspaces>
    <Keyspace Name="Keyspace1">
      <ColumnFamily Name="Super1"
                    ColumnType="Super"
                    CompareWith="BytesType"
                    CompareSubcolumnsWith="BytesType" />
      <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
      <ReplicationFactor>1</ReplicationFactor>
      <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
    </Keyspace>
  </Keyspaces>
  <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>

  <DiskAccessMode>auto</DiskAccessMode>
  <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
  <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
  <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
  <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
  <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>

  <MemtableThroughputInMB>16</MemtableThroughputInMB>
  <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
  <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
  <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
  <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
  <ConcurrentReads>4</ConcurrentReads>
  <ConcurrentWrites>8</ConcurrentWrites>
</Storage>


[2]
 INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
position=5417524)
 INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
 INFO 13:36:41,062 Writing Memtable(Super1)@15385755
 INFO 13:36:42,062 Completed flushing d:\cassandra\data\Keyspace1\Super1-711-Data.db
 INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
position=6065637)
 INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
 INFO 13:36:45,796 Writing Memtable(Super1)@15578910
 INFO 13:36:46,109 Completed flushing d:\cassandra\data\Keyspace1\Super1-712-Data.db
 INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed leaving 922392600
used; max is 1174208512
 INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh Memtable at CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
position=6722241)
 INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
 INFO 13:36:54,593 Writing Memtable(Super1)@24468872
 INFO 13:36:55,421 Completed flushing d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
Java heap space
 INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed leaving 971904520 used;
max is 1174208512



Mime
View raw message