incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Labrot <nith...@gmail.com>
Subject Re: Cassandra tuning for running test on a desktop
Date Wed, 21 Apr 2010 12:46:39 GMT
I have try 1400M, and Cassandra OOM too.

Is there another solution ? My data isn't very big.

It seems that is the merge of the db


On Wed, Apr 21, 2010 at 2:14 PM, Mark Greene <greenemj@gmail.com> wrote:

> Trying increasing Xmx. 1G is probably not enough for the amount of inserts
> you are doing.
>
>
> On Wed, Apr 21, 2010 at 8:10 AM, Nicolas Labrot <nithril@gmail.com> wrote:
>
>> Hello,
>>
>> For my first message I will first thanks Cassandra contributors for their
>> great works.
>>
>> I have a parameter issue with Cassandra (I hope it's just a parameter
>> issue). I'm using Cassandra 6.0.1 with Hector client on my desktop. It's a
>> simple dual core with 4GB of RAM on WinXP. I have keep the default JVM
>> option inside cassandra.bat (Xmx1G)
>>
>> I'm trying to insert 3 millions of SC with 6 Columns each inside 1 CF
>> (named Super1). The insertion go to 1 millions of SC (without slowdown) and
>> Cassandra crash because of an OOM. (I store an average of 100 bytes per SC
>> with a max of 10kB).
>> I have aggressively decreased all the memories parameters without any
>> respect to the consistency (My config is here [1]), the cache is turn off
>> but Cassandra still go to OOM. I have joined the last line of the Cassandra
>> life [2].
>>
>> What can I do to fix my issue ?  Is there another solution than increasing
>> the Xmx ?
>>
>> Thanks for your help,
>>
>> Nicolas
>>
>>
>>
>>
>>
>> [1]
>>   <Keyspaces>
>>     <Keyspace Name="Keyspace1">
>>       <ColumnFamily Name="Super1"
>>                     ColumnType="Super"
>>                     CompareWith="BytesType"
>>                     CompareSubcolumnsWith="BytesType" />
>>
>> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>>       <ReplicationFactor>1</ReplicationFactor>
>>
>> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
>>     </Keyspace>
>>   </Keyspaces>
>>   <CommitLogRotationThresholdInMB>32</CommitLogRotationThresholdInMB>
>>
>>   <DiskAccessMode>auto</DiskAccessMode>
>>   <RowWarningThresholdInMB>64</RowWarningThresholdInMB>
>>   <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>
>>   <FlushDataBufferSizeInMB>16</FlushDataBufferSizeInMB>
>>   <FlushIndexBufferSizeInMB>4</FlushIndexBufferSizeInMB>
>>   <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>
>>
>>   <MemtableThroughputInMB>16</MemtableThroughputInMB>
>>   <BinaryMemtableThroughputInMB>32</BinaryMemtableThroughputInMB>
>>   <MemtableOperationsInMillions>0.01</MemtableOperationsInMillions>
>>   <MemtableObjectCountInMillions>0.01</MemtableObjectCountInMillions>
>>   <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>
>>   <ConcurrentReads>4</ConcurrentReads>
>>   <ConcurrentWrites>8</ConcurrentWrites>
>> </Storage>
>>
>>
>> [2]
>>  INFO 13:36:41,062 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=5417524)
>>  INFO 13:36:41,062 Enqueuing flush of Memtable(Super1)@15385755
>>  INFO 13:36:41,062 Writing Memtable(Super1)@15385755
>>  INFO 13:36:42,062 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-711-Data.db
>>  INFO 13:36:45,781 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=6065637)
>>  INFO 13:36:45,781 Enqueuing flush of Memtable(Super1)@15578910
>>  INFO 13:36:45,796 Writing Memtable(Super1)@15578910
>>  INFO 13:36:46,109 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-712-Data.db
>>  INFO 13:36:54,296 GC for ConcurrentMarkSweep: 7149 ms, 58337240 reclaimed
>> leaving 922392600 used; max is 1174208512
>>  INFO 13:36:54,593 Super1 has reached its threshold; switching in a fresh
>> Memtable at
>> CommitLogContext(file='d:/cassandra/commitlog\CommitLog-1271849783703.log',
>> position=6722241)
>>  INFO 13:36:54,593 Enqueuing flush of Memtable(Super1)@24468872
>>  INFO 13:36:54,593 Writing Memtable(Super1)@24468872
>>  INFO 13:36:55,421 Completed flushing
>> d:\cassandra\data\Keyspace1\Super1-713-Data.dbjava.lang.OutOfMemoryError:
>> Java heap space
>>  INFO 13:37:08,281 GC for ConcurrentMarkSweep: 5561 ms, 9432 reclaimed
>> leaving 971904520 used; max is 1174208512
>>
>>
>

Mime
View raw message