cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-3224) LeveledCompactionStrategy is too complacent
Date Mon, 19 Sep 2011 20:26:09 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-3224:
--------------------------------------

    Attachment: 3224-v4.txt

v4 adds more debug logging, which makes clear that the empty manifests being logged were from
other CFS.

caps L0 compaction candidates at 32 to avoid OOMing from pessimisting BF sizing.

picks L0 initial victim based on age of data rather than randomly.

> LeveledCompactionStrategy is too complacent
> -------------------------------------------
>
>                 Key: CASSANDRA-3224
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3224
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Brandon Williams
>            Assignee: Jonathan Ellis
>              Labels: compaction
>             Fix For: 1.0.0
>
>         Attachments: 3224-v2.txt, 3224-v3.txt, 3224-v4.txt, 3224.txt, system.log.bz2
>
>
> As the title says, it barely does anything.  I inserted 50G worth of data with 1G heap
and 99% overwrite ratio, and it only compacted twice:
> {noformat}
>  INFO [CompactionExecutor:1] 2011-09-16 22:29:54,572 CompactionTask.java (line 118) Compacting
[SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-1-Data.db')]
>  INFO [CompactionExecutor:1] 2011-09-16 22:29:58,606 CompactionTask.java (line 220) Compacted
to [/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db,].
 12,595,811 to 12,595,811 (~100% of original) bytes for 40,501 keys at 3.058122MBPS.  Time:
3,928ms.
>  INFO [CompactionExecutor:1] 2011-09-16 22:29:58,607 CompactionTask.java (line 222) CF
Total Bytes Compacted: 12,595,811
>  INFO [CompactionExecutor:3] 2011-09-16 22:29:58,889 CompactionTask.java (line 118) Compacting
[SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-3-Data.db')]
>  INFO [CompactionExecutor:3] 2011-09-16 22:30:06,900 CompactionTask.java (line 220) Compacted
to [/var/lib/cassandra/data/Keyspace1/Standard1-h-7-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-9-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-11-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-12-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-14-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-15-Data.db,].
 28,374,396 to 28,374,396 (~100% of original) bytes for 91,236 keys at 3.380379MBPS.  Time:
8,005ms.
>  INFO [CompactionExecutor:3] 2011-09-16 22:30:06,901 CompactionTask.java (line 222) CF
Total Bytes Compacted: 40,970,207
> {noformat}
> Resulting in the following levels:
> {noformat}
> L0: 4965
> L1: 6
> L2: 0
> L3: 0
> L4: 0
> L5: 0
> L6: 0
> L7: 0
> {noformat}
> This is obviously going to result in extremely poor read performance.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message