cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Brown (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7282) Faster Memtable map
Date Fri, 19 Sep 2014 05:09:37 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14140010#comment-14140010
] 

Jason Brown commented on CASSANDRA-7282:
----------------------------------------

bq. I hope $data_dir is a shared dir with all the state, not just the data

Yeah, yeah, I got some things right today :)

I should note that when I set mct back to 0.111, the cluster launches properly - admittedly,
I didn't actually do anything with the cluster after I saw it successfully came and stayed
up, but it's still alive six hours later. I have not investigated further.

I agree that mct should be relatively harmless on new cluster launch, but let's back up for
a moment - would we ever set mct that high in a "real world" cluster? I understand we're trying
to avoid flushing (and losing the opportunity to actually test what the patch does), but if
those kind of mct settings are so artificial, shouldn't we be testing with more realistic,
or, at least, less crazy-town, values? 

> Faster Memtable map
> -------------------
>
>                 Key: CASSANDRA-7282
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: profile.yaml, reads.svg, run1.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in our
memtables. Maintaining this is an O(lg(n)) operation; since the vast majority of users use
a hash partitioner, it occurs to me we could maintain a hybrid ordered list / hash map. The
list would impose the normal order on the collection, but a hash index would live alongside
as part of the same data structure, simply mapping into the list and permitting O(1) lookups
and inserts.
> I've chosen to implement this initial version as a linked-list node per item, but we
can optimise this in future by storing fatter nodes that permit a cache-line's worth of hashes
to be checked at once,  further reducing the constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message