incubator-directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael André Pearce (Updated) (JIRA) <>
Subject [jira] [Updated] (DIRECTMEMORY-9) Add a defragmentation mechanism
Date Tue, 03 Apr 2012 19:26:26 GMT


Michael André Pearce updated DIRECTMEMORY-9:


Ok so have the old memory buffer architecture working with merging memory and a defragmentation,
also includes support for high concurrency, unfortunately i cannot seem to unpick the recent
changes that completely changed the memory buffer and its way of handling bits as it seems
so wide spread in one hit, as such i supply the following solution if it can be introduced
the old memory buffer or if the recent major changes that were done across the board were
to be reverted I would be willing to supply a patch on the older. As this resolves not only
the merging memory allocator it also resolves the other important issues i leave it to the
group how to proceed.
> Add a defragmentation mechanism
> -------------------------------
>                 Key: DIRECTMEMORY-9
>                 URL:
>             Project: Apache DirectMemory
>          Issue Type: Task
>            Reporter: Raffaele P. Guidi
>              Labels: defrag,, defragmentation
>         Attachments:,
> Add a defragmentation mechanism 
> From the ML: (paliwalashish)
> >Will the offHeapMemoryBuffer get fragmented over time? Say after a
> couple thousand get/remove operations, will the off-heap have start
> having holes in the Buffer?
> (Me:)
> >It will, definitely. I had two solutions ready in my mind (that rely on having more
than one buffer active): 
> Simplest, and fastest but with some drawbacks: when buffer.isTooDefragmented() then simply
buffer.clear() - you loose everything, but - hey, it's a cache, not a db
> Less simple, slower, less drawbacks: when buffer.isTooDefragmented() mark the buffer
as readOnly and then foreach (ptr in buffer) copy ptr.content in emptyBuffer and update ptr
> where isTooFragmented==number_of_empty_pointers over total_pointers > desirable quota
> The first one could be accomplished during a put() operation (buffer.clear is a logical
operation that takes no time) while the second should be taken care of by the background thread.
Those quick&dirty solutions could of course be replaced with real defragmentation algorithms
- may taken from various malloc() implementations, that are the original inspiration
> See also

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message