cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takenori Sato (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-8866) PartitionedCompactionStrategy
Date Thu, 05 Mar 2015 00:46:38 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347849#comment-14347849
] 

Takenori Sato commented on CASSANDRA-8866:
------------------------------------------

{quote}
This certainly wouldn't work in the general case of millions to billions of partitions, but
I suppose it could be useful if you know you're only going to have ~thousands per node.
{quote}

I agree. I designed a partition as an equally divided range of a ring by a given IPartitioner.

I think the number of partitions is considered large enough if it puts hugest/hottest rows
separately in different partitions. Suppose that hugeness/hotness follows long tail distribution,
the number is expected around 10 ~ 100 in general.

> PartitionedCompactionStrategy
> -----------------------------
>
>                 Key: CASSANDRA-8866
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8866
>             Project: Cassandra
>          Issue Type: New Feature
>            Reporter: Takenori Sato
>         Attachments: cassandra-2.0-8866.txt
>
>
> PartitionedCompactionStrategy is a new compaction strategy with the following goals in
mind:
> * Column tombstone removal effectiveness
> * Read performance
> As the name suggests, PartitionedCompactionStrategy actively splits un-partitioned sstables(newly
flushed, imported, compaction strategy switch) into partitions by IPartitioner. The number
of nodes will be configurable.
> Then, PartitionedCompactionStrategy finds an interesting partition at compaction based
on the followings:
> - the number of sstables
> - the ratio of droppable tombstones
> - read hotness
> You may think this design looks similar to SizeTieredCompactionStrategy and LeveledCompactionStrategy,
but the big difference is that a compaction by PartitionedCompactionStrategy is based on rows(a
partitions). And this allows more effective column tombstone removal, and better read performance.
> Also note that this will not require any changes to the other components. So this is
expected to be a purely pluggable compaction strategy.
> A possible implementation of _PertitionedCompactionStrategy#getNextBackgroundTask()_
is as follows:
> # find un-partitioned sstables
> # split un-partitioned sstables into partitiones
> # group all the sstables into partitions
> # find an interesting partition
> #* the number of sstables
> #* the number of droppable tombstones
> #* hotness
> # create a compaction task for the interesting bucket if found



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message