cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jussi P?├Âri <>
Subject Inserting files to Cassandra timeouts
Date Wed, 28 Apr 2010 14:24:34 GMT
new try, previous went to wrong place...

Hi all,

i'm trying to run a scenario of adding files from specific folder to cassandra. Now I have
64 files(about 15-20 MB per file) and overall of 1GB of data. 
I'm able to insert a round 40 files, but after that the cassandra goes to some GC loop and
I finally get an timeout to the client. 
It is not going to OOM, but it just jams. 

Here is what I had last marks in log file:
NFO [GC inspection] 2010-04-28 10:07:55,297 (line 110) GC for ParNew: 232
ms, 25731128 reclaimed leaving 553241120 used; max is 4108386304
 INFO [GC inspection] 2010-04-28 10:09:02,331 (line 110) GC for ParNew: 2844
ms, 238909856 reclaimed leaving 1435582832 used; max is 4108386304
 INFO [GC inspection] 2010-04-28 10:09:49,421 (line 110) GC for ParNew: 30666
ms, 11185824 reclaimed leaving 1679795336 used; max is 4108386304
 INFO [GC inspection] 2010-04-28 10:11:18,090 (line 110) GC for ParNew: 895
ms, 17921680 reclaimed leaving 1589308456 used; max is 4108386304

I think that I must have something wrong in my configurations or in how I use cassandra, because
here people are inserting 10 times more stuff and it works. 

Column family I using:
<ColumnFamily CompareWith="BytesType" Name="Standard1"/>
Basically inserting with key name is "Folder_name" and column name is "file name" and value
is the file content. 
I tried with Hector(mainly) and directly using thrift(insert and batch_mutate). 

In my case, the data does not need to readable immediately after insert, but I don't know
it that helps in anyway. 

My environment :
mac and/or linux, tested in both
java 1.6.0_17 
Cassandra 0.6.1

  <!-- CommitLogSyncPeriodInMS>10000</CommitLogSyncPeriodInMS -->

        -server \
        -Xms3G \
        -Xmx3G \
        -XX:PermSize=512m \
        -XX:MaxPermSize=800m \
        -XX:MaxNewSize=256m \
        -XX:NewSize=128m \
        -XX:TargetSurvivorRatio=90 \
        -XX:+AggressiveOpts \
        -XX:+UseParNewGC \
        -XX:+UseConcMarkSweepGC \
        -XX:+CMSParallelRemarkEnabled \
        -XX:+HeapDumpOnOutOfMemoryError \
        -XX:SurvivorRatio=128 \
        -XX:MaxTenuringThreshold=0 \
        -XX:+DisableExplicitGC \ \ \"

View raw message