commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raffaele P. Guidi" <raffaele.p.gu...@gmail.com>
Subject Re: [JCS] - proposal on a off-heap plugin for JCS based on DirectMemory
Date Tue, 08 Nov 2011 23:10:48 GMT
Cool. Basically the plugin should fit in the middle between standard heap
cache and the lateral (network) plugin. DirectMemory exposes two basic
services: the memorystorage, which acts almost like a malloc(), allocating
large buffers which can be split in "slices" described by a Pointer
structure (with a start and a length). The (higher level) Cache facade
allocates memory using the MemoryStorage and saves the pointers into a map
handling transparent serialization of objects (using protostuff).

This is not so different from an indexed file, just it is in memory - maybe
you could take a look at the code (which is fairly simple) and get back to
us with some more targeted questions.

Just to recap a bit the rationale behind it: direct memory is supposed to
be a bit slower than heap (because objects need to be serialized and
unserialized back) but way faster than the file system and - most of all -
it is not garbage collected, thus removing the pain of gc pauses even using
 huge buffers (even of several GBs) in memory.  Here is some reference on
the matter from the original project wiki [1], my blog [2] and terracotta
bigmemory product pages [3].

https://github.com/raffaeleguidi/DirectMemory/wiki
http://raffaeleguidi.wordpress.com/2011/01/12/directmemory-benchmark-heap-vs-off-heap-vs-orientdb/
http://www.terracotta.org/products/bigmemory

Ciao,
    R

On Tue, Nov 8, 2011 at 9:39 PM, Thomas Vandahl <tv@apache.org> wrote:

> On 08.11.11 20:59, Raffaele P. Guidi wrote:
> > Are you guys interested in merging efforts on this?
>
> Yes, absolutely. I, however, would need some introduction into the
> subject to understand how this is supposed to work. I can assure my
> support for the integration into the JCS code.
>
> Bye, Thomas.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message