incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brandon Williams <dri...@gmail.com>
Subject Re: memory issue on 1.1.0
Date Tue, 05 Jun 2012 04:24:01 GMT
Perhaps the deletes: https://issues.apache.org/jira/browse/CASSANDRA-3741

-Brandon

On Sun, Jun 3, 2012 at 6:12 PM, Poziombka, Wade L
<wade.l.poziombka@intel.com> wrote:
> Running a very write intensive (new column, delete old column etc.) process and failing
on memory.  Log file attached.
>
> Curiously when I add new data I have never seen this have in past sent hundreds of millions
"new" transactions.  It seems to be when I modify.  my process is as follows
>
> key slice to get columns to modify in batches of 100, in separate threads modify those
columns.  I advance the slice with the start key each with last key in previous batch.  Mutations
done are update a column value in one column family(token), delete column and add new column
in another (pan).
>
> Runs well until after about 5 million rows then it seems to run out of memory.  Note
that these column families are quite small.
>
> WARN [ScheduledTasks:1] 2012-06-03 17:49:01,558 GCInspector.java (line 145) Heap is 0.7967470834946492
full.  You may need to reduce memtable and/or cache sizes.  Cassandra will now flush up
to the two largest memtables to free up memory.  Adjust flush_largest_memtables_at threshold
in cassandra.yaml if you don't want Cassandra to do this automatically
>  INFO [ScheduledTasks:1] 2012-06-03 17:49:01,559 StorageService.java (line 2772) Unable
to reduce heap usage since there are no dirty column families
>  INFO [GossipStage:1] 2012-06-03 17:49:01,999 Gossiper.java (line 797) InetAddress /10.230.34.170
is now UP
>  INFO [ScheduledTasks:1] 2012-06-03 17:49:10,048 GCInspector.java (line 122) GC for
ParNew: 206 ms for 1 collections, 7345969520 used; max is 8506048512
>  INFO [ScheduledTasks:1] 2012-06-03 17:49:53,187 GCInspector.java (line 122) GC for
ConcurrentMarkSweep: 12770 ms for 1 collections, 5714800208 used; max is 8506048512
>
> ----------------
> Keyspace: keyspace
>        Read Count: 50042632
>        Read Latency: 0.23157864418482224 ms.
>        Write Count: 44948323
>        Write Latency: 0.019460829472992797 ms.
>        Pending Tasks: 0
>                Column Family: pan
>                SSTable count: 5
>                Space used (live): 1977467326
>                Space used (total): 1977467326
>                Number of Keys (estimate): 16334848
>                Memtable Columns Count: 0
>                Memtable Data Size: 0
>                Memtable Switch Count: 74
>                Read Count: 14985122
>                Read Latency: 0.408 ms.
>                Write Count: 19972441
>                Write Latency: 0.022 ms.
>                Pending Tasks: 0
>                Bloom Filter False Postives: 829
>                Bloom Filter False Ratio: 0.00073
>                Bloom Filter Space Used: 37048400
>                Compacted row minimum size: 125
>                Compacted row maximum size: 149
>                Compacted row mean size: 149
>
>                Column Family: token
>                SSTable count: 4
>                Space used (live): 1250973873
>                Space used (total): 1250973873
>                Number of Keys (estimate): 14217216
>                Memtable Columns Count: 0
>                Memtable Data Size: 0
>                Memtable Switch Count: 49
>                Read Count: 30059563
>                Read Latency: 0.167 ms.
>                Write Count: 14985488
>                Write Latency: 0.014 ms.
>                Pending Tasks: 0
>                Bloom Filter False Postives: 13642
>                Bloom Filter False Ratio: 0.00322
>                Bloom Filter Space Used: 28002984
>                Compacted row minimum size: 150
>                Compacted row maximum size: 258
>                Compacted row mean size: 224
>
>                Column Family: counters
>                SSTable count: 2
>                Space used (live): 561549994
>                Space used (total): 561549994
>                Number of Keys (estimate): 9985024
>                Memtable Columns Count: 0
>                Memtable Data Size: 0
>                Memtable Switch Count: 38
>                Read Count: 4997947
>                Read Latency: 0.092 ms.
>                Write Count: 9990394
>                Write Latency: 0.023 ms.
>                Pending Tasks: 0
>                Bloom Filter False Postives: 191
>                Bloom Filter False Ratio: 0.37525
>                Bloom Filter Space Used: 18741152
>                Compacted row minimum size: 125
>                Compacted row maximum size: 179
>                Compacted row mean size: 150
>
> ----------------

Mime
View raw message