cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables
Date Wed, 05 Mar 2014 01:23:47 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920357#comment-13920357
] 

Benedict commented on CASSANDRA-6689:
-------------------------------------

bq. So the only question that I still have - what is the main goal of the ticket? Is it to
move to zero copy for memtables or minimize number of allocations on java heap?

Zero copy would certainly be nice to retain. Otherwise why not just ref-count the memory?

bq.  allocator interface is going to be better encapsulated (basically expose alloc/release)

I don't follow how this would improve encapsulation, sorry. Could you sketch an example? It
sounds to me like this actually would create more complexity outside of the main body of code.
It might reduce complexity a little where the fun stuff happens, but is it all really so difficult
to understand that we want to start opting for less good solutions that require more work?
Also makes it difficult to impose tight caps on memory used, as we can't practicably impose/manage
temporary memory allocated by readers. We already have this problem for disk reads, but no
point opening up further doors for OOM.

bq.  I'm not sure if we need sub pools or groups of allocators in there, I really like how
jemalloc and others do it also Netty already has implementation of jemalloc (ByteBufAlloc)
so maybe we can reuse that?...

I'm unclear how this addresses the concerns of resource management. The ByteBufAllocator appears
to simply allocate byte buffers, which by itself is a pretty simple problem. jemalloc is also
just a low level replacement to malloc, so I'm not sure where it enters the discussion?

> Partially Off Heap Memtables
> ----------------------------
>
>                 Key: CASSANDRA-6689
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>             Fix For: 2.1 beta2
>
>         Attachments: CASSANDRA-6689-small-changes.patch
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message