cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables
Date Fri, 25 Apr 2014 09:43:22 GMT


Benedict commented on CASSANDRA-6694:

On the whole it looks good, but I have the following comments/concerns:

# DecoratedKey still isn't implemented (should be a relatively minor addition)
# The performance regression for MessageDigest updating is still there
# AbstractCell.localCopy(..MemtableAllocator) needs to be overridden; as it is you'll always
get a regular Cell back
# You're still using static method implementations, it looks like? Cell.diff and Cell.reconcile
# I'm not a fan of mixing the util.memory hierarchy with knowledge of the memtable hierarchy.
If we plan on this, I'd much prefer to move the whole lot into e.g. db.memtable; this might
make most sense anyway
# I'd like to move the Cell implementations out of db into something (e.g. .memtable) as it's
very crowded in there, and they're a dozen or so related classes that are easily extracted
# Given how many different kinds of allocator we now have (including IAllocator), I'd really
like to rename AbstractAllocator to something more descriptive like ByteBufferAllocator

Still need to verify all of the changes within the Cells, as comparison is currently tricky
due to different hierarchy confusing git, but on the whole this branch is good if we address
these concerns.

> Slightly More Off-Heap Memtables
> --------------------------------
>                 Key: CASSANDRA-6694
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>              Labels: performance
>             Fix For: 2.1 beta2
> The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as the on-heap
overhead is still very large. It should not be tremendously difficult to extend these changes
so that we allocate entire Cells off-heap, instead of multiple BBs per Cell (with all their
associated overhead).
> The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 bytes per
cell on average for the btree overhead, for a total overhead of around 20-22 bytes). This
translates to 8-byte object overhead, 4-byte address (we will do alignment tricks like the
VM to allow us to address a reasonably large memory space, although this trick is unlikely
to last us forever, at which point we will have to bite the bullet and accept a 24-byte per
cell overhead), and 4-byte object reference for maintaining our internal list of allocations,
which is unfortunately necessary since we cannot safely (and cheaply) walk the object graph
we allocate otherwise, which is necessary for (allocation-) compaction and pointer rewriting.
> The ugliest thing here is going to be implementing the various CellName instances so
that they may be backed by native memory OR heap memory.

This message was sent by Atlassian JIRA

View raw message