Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Thu, 17 Apr 2014 23:56:18 +0000 (UTC)
From: "Benedict (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12694748.1392215110539.136202.1397778978881@arcas>
In-Reply-To: <JIRA.12694748.1392215110539@arcas>
References: <JIRA.12694748.1392215110539@arcas>
Subject: [jira] [Comment Edited] (CASSANDRA-6694) Slightly More Off-Heap
 Memtables
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973582#comment-13973582 ] 

Benedict edited comment on CASSANDRA-6694 at 4/17/14 11:55 PM:
---------------------------------------------------------------

The only reason we were assigning a size in NativeAllocator was to support moving the peer around (in which case you need to know how much memory you're copying). 

NativeAllocation assuming it has (i.e. _being defined as having_) a size prefix is fine when it is tightly coupled with NativeAllocator (like it is in my branch) - but once you have it as a final field in another object, NativeAllocator should simply have no say in the matter. It never needs to know the size of the allocation, so we should just redefine what our AbstractNativeCell considers to be its size in its sizeOf() calculation, and have the NativeAllocator use that unadulterated value.


was (Author: benedict):
The only reason it was happening in NativeAllocator was to support moving the peer around (so you need to know how much memory you're copying). 

NativeAllocation assuming it has (i.e. _being defined as having_) a size prefix is fine when it is tightly coupled with NativeAllocator (like it is in my branch) - but once you have it as a final field in another object, NativeAllocator should simply have no say in the matter. It never needs to know the size of the allocation, so we should just redefine what our AbstractNativeCell considers to be its size in its sizeOf() calculation.

> Slightly More Off-Heap Memtables
> --------------------------------
>
>                 Key: CASSANDRA-6694
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>              Labels: performance
>             Fix For: 2.1 beta2
>
>
> The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as the on-heap overhead is still very large. It should not be tremendously difficult to extend these changes so that we allocate entire Cells off-heap, instead of multiple BBs per Cell (with all their associated overhead).
> The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 bytes per cell on average for the btree overhead, for a total overhead of around 20-22 bytes). This translates to 8-byte object overhead, 4-byte address (we will do alignment tricks like the VM to allow us to address a reasonably large memory space, although this trick is unlikely to last us forever, at which point we will have to bite the bullet and accept a 24-byte per cell overhead), and 4-byte object reference for maintaining our internal list of allocations, which is unfortunately necessary since we cannot safely (and cheaply) walk the object graph we allocate otherwise, which is necessary for (allocation-) compaction and pointer rewriting.
> The ugliest thing here is going to be implementing the various CellName instances so that they may be backed by native memory OR heap memory.


--
This message was sent by Atlassian JIRA
(v6.2#6252)