cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Morishita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5906) Avoid allocating over-large bloom filters
Date Fri, 27 Sep 2013 01:52:02 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779565#comment-13779565
] 

Yuki Morishita commented on CASSANDRA-5906:
-------------------------------------------

v2 pushed to: https://github.com/yukim/cassandra/commits/5906-v2

HLL++ support for out-of-library hashing is merged(https://github.com/clearspring/stream-lib/pull/50),
but not officially released yet, so v2 contains custom built stream-lib.
HLL++ parameter is unchanged from first version.

Other change I made was to let SSTableMetadata to support reading HLL++ on compaction time
only. To make this change a little bit easier, I also rewrite mutateLevel not to deserialize
SSTableMetadata object(https://github.com/yukim/cassandra/commit/016c89b68ba74ca15fcac9fa6e6c37faeaee7bcd).
                
> Avoid allocating over-large bloom filters
> -----------------------------------------
>
>                 Key: CASSANDRA-5906
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5906
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Yuki Morishita
>             Fix For: 2.0.2
>
>
> We conservatively estimate the number of partitions post-compaction to be the total number
of partitions pre-compaction.  That is, we assume the worst-case scenario of no partition
overlap at all.
> This can result in substantial memory wasted in sstables resulting from highly overlapping
compactions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message