cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Favre-Felix (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4482) In-memory merkle trees for repair
Date Tue, 14 Aug 2012 13:20:38 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434105#comment-13434105
] 

Nicolas Favre-Felix commented on CASSANDRA-4482:
------------------------------------------------

bq. Meaning, you give each CF less than 64k ranges * 16 bytes / range?

Right, that would be too much. At the moment, we give each CF 256 KB to be split into all
of its ranges. For num_tokens=256, that's 1 KB per range on average - we do not yet scale
this number according to the range size.

A node with num_tokens = 1 owning a single range would allocate 256 KB in a single direct
ByteBuffer. Moving to num_tokens = 256 gives the ColumnFamilyStore 256 ranges, and allocates
a 1 KB ByteBuffer per range. In both cases the keys in any given range are covered by as many
"leaf bytes" on average, regardless of the number of ranges.

bq. Is there a startup cost associated with the approach? i.e. How to you know the initial
hash?

We do have to reload $num_tokens ByteBuffers when creating the ColumnFamilyStore, for a total
of 256KB per CF with our current defaults. This is not something we've measured but I suspect
that the cost is fairly small, as it is now for the cache snapshots: it is O(number of CFs),
not O(N) like the old cache preloads.
                
> In-memory merkle trees for repair
> ---------------------------------
>
>                 Key: CASSANDRA-4482
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4482
>             Project: Cassandra
>          Issue Type: New Feature
>            Reporter: Marcus Eriksson
>
> this sounds cool, we should reimplement it in the open source cassandra;
> http://www.acunu.com/2/post/2012/07/incremental-repair.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message