cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables
Date Wed, 12 Feb 2014 13:42:20 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899096#comment-13899096
] 

Benedict commented on CASSANDRA-6689:
-------------------------------------

There are some natural boundaries if you want to share the burden. Everything inside of utils.concurrent
is pretty isolated from everything outside, so could easily be vetted independently.

Also, the utilisation of Referrer/RefAction is probably going to be a painstaking thing to
vet (that's what makes the majority of small touches outside of the main changes), and quite
independent of their declarations. We just need to be certain we always use the correct type
of RefAction, and never let one disappear somewhere - OutboundTCPConnection and native transport
writing are the two danger areas here (also Memtable flushing needs a bit of care, but is
definitely less scary).

The most difficult thing to review is going to be the main body of work inside of utils.memory,
however. This is pretty hardcore lock-free stuff, and the thing we're looking for is _unintended_
race conditions (there are lots of intended races) - in particular pay attention to the way
in which we now asynchronously manage the "subpool" and "suballocator" (ledgers of how much
we've allocated / claimed / are reclaiming), and obviously most importantly that we never
accidentally overwrite data that is being read elsewhere. This should all hopefully be very
clearly documented both at the level of abstraction and the individual points where interesting
/ dangerous things happen. But try to figure it out for yourself as well, in case I and my
tests missed something. I will be doing further tests in the near future, but I much prefer
to catch things by eye if possible.

Always feel free to throw up a "this bit isn't well explained" flag and I'll try to improve
it. I want this stuff to be as clearly self documenting as possible.



> Partially Off Heap Memtables
> ----------------------------
>
>                 Key: CASSANDRA-6689
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>             Fix For: 2.1
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message