cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables
Date Thu, 27 Feb 2014 19:13:20 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13914908#comment-13914908
] 

Jonathan Ellis commented on CASSANDRA-6689:
-------------------------------------------

For the reviewers, some high level context might help:

We want to reduce the amount of space memtables use on the java heap,
both for the DSE in-memory tables and because that allows C* proper to
reduce flush + compact write amplification.

Since we already do arena allocation, it occurred to us long ago that
it would be nice to allocate those arenas off heap.  The problem is
that the refcounting overhead to make sure we don't free a flushed
memtable before all reads are finished with it gets really ugly.

CASSANDRA-5549 introduced the concept of an OpOrder.  This lets us
track memtable lifetime at a much coarser level (i.e., less overhead)
while still allowing us to be confident that we know when it's safe to
free.

So, 6689 takes the first step of "let's make the ByteBuffers of a cell
point to off-heap arenas."  But this still leaves the ByteBuffers
themselves on-heap which is significant.  6694 introduces NativeCell
which rolls the cell name and value (and ts) into a single off-heap
area, so your on-heap cost is cut to basically the NC itself and a
pointer.

> Partially Off Heap Memtables
> ----------------------------
>
>                 Key: CASSANDRA-6689
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>             Fix For: 2.1 beta2
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message