cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-7282) Faster Memtable map
Date Fri, 12 Sep 2014 19:49:34 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benedict updated CASSANDRA-7282:
--------------------------------
    Attachment: reads.svg
                writes.svg

Okay, so I made some minor tweaks to this and did some more serious local testing (and graphing
of results). The updated branch (also rebased to trunk) is [here|https://github.com/belliottsmith/cassandra/tree/7282-fastmemtablemap.trunk]

On the whole it looks to me that bdplab is as usual showing its reticence to exhibit performance
improvements. I suspect it's bottlenecking more readily on kernel operations.

On my local machine I see around a 15-20% improvement in throughput for both reads and writes,
making this a very sensible addition. It also sees considerably _reduced_ GC time (though
not amount of garbage generated) on writes, and sees a reduction in latency almost across
the board (max latency on a read-only workload is slightly bumped, but since write workloads
have the largest effect on latency, this seems worth overlooking, and due to how closely run
we are, it's possible this is noise in the measurement, since the median p999/pMax are almost
exactly the same)

I've also separately improved stress to collect GC data over JMX and created a patch to generate
these pretty graphs from stress output automatically, which I'll be posting separately.

> Faster Memtable map
> -------------------
>
>                 Key: CASSANDRA-7282
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: reads.svg, writes.svg
>
>
> Currently we maintain a ConcurrentSkipLastMap of DecoratedKey -> Partition in our
memtables. Maintaining this is an O(lg(n)) operation; since the vast majority of users use
a hash partitioner, it occurs to me we could maintain a hybrid ordered list / hash map. The
list would impose the normal order on the collection, but a hash index would live alongside
as part of the same data structure, simply mapping into the list and permitting O(1) lookups
and inserts.
> I've chosen to implement this initial version as a linked-list node per item, but we
can optimise this in future by storing fatter nodes that permit a cache-line's worth of hashes
to be checked at once,  further reducing the constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message