cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CASSANDRA-2252) off-heap memtables
Date Sun, 27 Feb 2011 03:59:59 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stu Hood updated CASSANDRA-2252:
--------------------------------

    Attachment: 2252-v1.tgz

We've been developing a very similar patch in parallel: attached as 2252-alternate-v1.tgz.

The key differences seem to be:
* Original removes interning from deserialization, alternate encapsulates it
* Alternate has support for allocating reconciled counter columns (the result of merging counters)
in slabs
* Original uses JNA for direct memory access, alternate uses ByteBuffer.allocateDirect
* Alternate uses the allocation count of the Memtable's allocator to measure throughput for
thresholds (less accurate, but necessary to track the true memory usage of counters)

We've tested the alternate patch in a production setting with counter mutations to multiple
column families: it reduced our fragmentation enough to double the time between promotion
failures, but it does not fully eliminate fragmentation.

> off-heap memtables
> ------------------
>
>                 Key: CASSANDRA-2252
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Assignee: Jonathan Ellis
>             Fix For: 0.8
>
>         Attachments: 0001-add-MemtableAllocator.txt, 0002-add-off-heap-MemtableAllocator-support.txt,
2252-alternate-v1.tgz
>
>
> The memtable design practically actively fights Java's GC design.  Todd Lipcon gave a
good explanation over on HBASE-3455.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message