incubator-directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raffaele P. Guidi (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DIRECTMEMORY-63) Investigate removing the need for an on heap map to store keys/pointers.
Date Sun, 19 Feb 2012 09:14:34 GMT

    [ https://issues.apache.org/jira/browse/DIRECTMEMORY-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211291#comment-13211291
] 

Raffaele P. Guidi commented on DIRECTMEMORY-63:
-----------------------------------------------

Well, the general idea was, from the beginning, to keep "indexes" into the heap and data outside
the heap; I've made attempts at keeping the "index" part (pointers and keys) as thin as possible
and come up with Guava collections as the better tradeoff. I also tried using CRC32 instead
of keys but didn't work out well (too hard to calculate). I meant to give Vanilla HugeCollections
(http://code.google.com/p/vanilla-java/wiki/HugeCollections - off-heap collections of homogeneus
objects) a try to make the Pointer's part even lighter but still havent' had a chance to play
with it (do someone knows the thing?). Of course the most heavyweight part is now the key
itself and maybe it's worth a bit of hacking
                
> Investigate removing the need for an on heap map to store keys/pointers.
> ------------------------------------------------------------------------
>
>                 Key: DIRECTMEMORY-63
>                 URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-63
>             Project: Apache DirectMemory
>          Issue Type: Improvement
>            Reporter: Michael André Pearce
>
> On performance testing and monitoring jvm heap allocation, it became clear that as the
off heap elements increased so did the on heap used memory. This is because DirectMemory is
storing keys and pointers in the heap meaning that the cache isn't fully off heap, and on
heap memory will increase linearly with elements in the cache.
> An idea maybe is to look into using a key's hash for location on a separate single off
heap buffer to store the main heaps pointers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message