jackrabbit-oak-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Mueller <muel...@adobe.com>
Subject Re: Design idea for a production-scale in-memory microkernel
Date Mon, 13 Aug 2012 07:56:04 GMT
Hi,

I think the problem you want to solve is having a high performance storage
subsystem. There are multiple ways to solve this.

One is to write a new in-memory microkernel. Another is to use a very big
cache, and write in a background thread. This would also work if the data
doesn't fit in memory. Also, it might be simpler to implement than writing
a new in-memory microkernel, as this would also need a way to write to and
read from disk, which is something we already have.

Never writing to disk, and just relying on other cluster nodes for
redundancy (replicating the data over the network), might be a bit
problematic, for example for maintenance: shutdown and startup time would
be proportional to the size of the repository. To protect against power
failures, the cluster nodes would need to be quite far apart, and if this
is the case, using 'the network as the storage' might not be much faster
than using a disk.

In any case, if we want to keep a lot of data (multiple GB) in memory
(whether it's a large cache or the main data area), we should consider
using non-heap / off-heap memory. There are multiple ways to do it, one is
using http://wiki.apache.org/incubator/DirectMemoryProposal


Regards,
Thomas


Mime
View raw message