incubator-directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benoit Perroud (Commented) (JIRA)" <>
Subject [jira] [Commented] (DIRECTMEMORY-9) Add a defragmentation mechanism
Date Wed, 28 Dec 2011 09:35:30 GMT


Benoit Perroud commented on DIRECTMEMORY-9:

Regarding the second idea, a simple idea that is a sort of trade off between both is the following
Instead of copying all the content into another buffer and clearing the too fragmented buffer,
we could implement a mechanism to set the too fragmented buffer as read only. Then every read
and remove will still be fine, but update will remove the value from the read only buffer
and write it into another not read only buffer. Then once the buffer is no more too fragmented,
it could re-enable write. Or be completely cleared after some time. The behavior should be
> Add a defragmentation mechanism
> -------------------------------
>                 Key: DIRECTMEMORY-9
>                 URL:
>             Project: Apache DirectMemory
>          Issue Type: Task
>            Reporter: Raffaele P. Guidi
>              Labels: defrag,, defragmentation
> Add a defragmentation mechanism 
> From the ML: (paliwalashish)
> >Will the offHeapMemoryBuffer get fragmented over time? Say after a
> couple thousand get/remove operations, will the off-heap have start
> having holes in the Buffer?
> (Me:)
> >It will, definitely. I had two solutions ready in my mind (that rely on having more
than one buffer active): 
> Simplest, and fastest but with some drawbacks: when buffer.isTooDefragmented() then simply
buffer.clear() - you loose everything, but - hey, it's a cache, not a db
> Less simple, slower, less drawbacks: when buffer.isTooDefragmented() mark the buffer
as readOnly and then foreach (ptr in buffer) copy ptr.content in emptyBuffer and update ptr
> where isTooFragmented==number_of_empty_pointers over total_pointers > desirable quota
> The first one could be accomplished during a put() operation (buffer.clear is a logical
operation that takes no time) while the second should be taken care of by the background thread.
Those quick&dirty solutions could of course be replaced with real defragmentation algorithms
- may taken from various malloc() implementations, that are the original inspiration
> See also

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message