cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Williams (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3224) LeveledCompactionStrategy is too complacent
Date Mon, 19 Sep 2011 17:45:08 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108018#comment-13108018
] 

Brandon Williams commented on CASSANDRA-3224:
---------------------------------------------

Better, but it looks like LCS starts miscounting at some point, and this causing compactions
to stop:

{noformat}

DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,359 LeveledManifest.java (line 218) Level
0 contains 85 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,360 LeveledManifest.java (line 218) Level
1 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,360 LeveledManifest.java (line 218) Level
2 contains 14 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line 218) Level
3 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line 218) Level
4 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line 218) Level
5 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line 218) Level
6 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line 218) Level
7 contains 0 SSTables
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,361 LeveledManifest.java (line 197) Compaction
score for level 0 is 0.0
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,362 LeveledManifest.java (line 197) Compaction
score for level 2 is 0.0
DEBUG [CompactionExecutor:4] 2011-09-19 17:40:31,362 LeveledCompactionStrategy.java (line
112) CompactionManager candidates are 
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line 218) Level
0 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line 218) Level
1 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line 218) Level
2 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,362 LeveledManifest.java (line 218) Level
3 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line 218) Level
4 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line 218) Level
5 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line 218) Level
6 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line 218) Level
7 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledCompactionStrategy.java (line
112) CompactionManager candidates are 
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line 218) Level
0 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line 218) Level
1 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,363 LeveledManifest.java (line 218) Level
2 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line 218) Level
3 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line 218) Level
4 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line 218) Level
5 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line 218) Level
6 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledManifest.java (line 218) Level
7 contains 0 SSTables
DEBUG [CompactionExecutor:1] 2011-09-19 17:40:31,364 LeveledCompactionStrategy.java (line
112) CompactionManager candidates are 
{noformat}

Actual levels are (according to json manifest):
{noformat}
L0: 85
L1: 0
L2: 14
L3: 0
L4: 0
L5: 0
L6: 0
L7: 0
{noformat}

Attempting to force a compaction via nodetool:
{noformat}

 INFO 17:42:28,933 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db'),
SSTableReader(path='/var/lib
/cassandra/data/system/LocationInfo-h-1-Data.db')]
DEBUG 17:42:28,933 Expected bloom filter size : 512
 INFO 17:42:28,956 Compacted to [/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db,].
 498 to 447 (~89% of original) bytes for 3 key
s at 0.019377MBPS.  Time: 22ms.
 INFO 17:42:28,956 CF Total Bytes Compacted: 225,632,757
 INFO 17:42:28,956 Nothing to compact in Migrations.Use forceUserDefinedCompaction if you
wish to force compaction of single sstables (e.g. for tombstone collection)
 INFO 17:42:28,957 Nothing to compact in Schema.Use forceUserDefinedCompaction if you wish
to force compaction of single sstables (e.g. for tombstone collection)
{noformat}

> LeveledCompactionStrategy is too complacent
> -------------------------------------------
>
>                 Key: CASSANDRA-3224
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3224
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Brandon Williams
>            Assignee: Jonathan Ellis
>              Labels: compaction
>             Fix For: 1.0.0
>
>         Attachments: 3224.txt
>
>
> As the title says, it barely does anything.  I inserted 50G worth of data with 1G heap
and 99% overwrite ratio, and it only compacted twice:
> {noformat}
>  INFO [CompactionExecutor:1] 2011-09-16 22:29:54,572 CompactionTask.java (line 118) Compacting
[SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-1-Data.db')]
>  INFO [CompactionExecutor:1] 2011-09-16 22:29:58,606 CompactionTask.java (line 220) Compacted
to [/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db,].
 12,595,811 to 12,595,811 (~100% of original) bytes for 40,501 keys at 3.058122MBPS.  Time:
3,928ms.
>  INFO [CompactionExecutor:1] 2011-09-16 22:29:58,607 CompactionTask.java (line 222) CF
Total Bytes Compacted: 12,595,811
>  INFO [CompactionExecutor:3] 2011-09-16 22:29:58,889 CompactionTask.java (line 118) Compacting
[SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-4-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-5-Data.db'), SSTableReader(path='/var/lib/cassandra/data/Keyspace1/Standard1-h-3-Data.db')]
>  INFO [CompactionExecutor:3] 2011-09-16 22:30:06,900 CompactionTask.java (line 220) Compacted
to [/var/lib/cassandra/data/Keyspace1/Standard1-h-7-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-9-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-11-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-12-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-14-Data.db,/var/lib/cassandra/data/Keyspace1/Standard1-h-15-Data.db,].
 28,374,396 to 28,374,396 (~100% of original) bytes for 91,236 keys at 3.380379MBPS.  Time:
8,005ms.
>  INFO [CompactionExecutor:3] 2011-09-16 22:30:06,901 CompactionTask.java (line 222) CF
Total Bytes Compacted: 40,970,207
> {noformat}
> Resulting in the following levels:
> {noformat}
> L0: 4965
> L1: 6
> L2: 0
> L3: 0
> L4: 0
> L5: 0
> L6: 0
> L7: 0
> {noformat}
> This is obviously going to result in extremely poor read performance.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message