Mailing-List: contact dev-help@directmemory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@directmemory.apache.org
Received-SPF: pass (nike.apache.org: domain of kjan80@gmail.com designates
 209.85.215.178 as permitted sender)
Message-ID: <509E89EB.6000303@gmail.com>
Date: Sat, 10 Nov 2012 17:07:55 +0000
From: Jan Kotek <kjan80@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:16.0) Gecko/20121028 Thunderbird/16.0.2
MIME-Version: 1.0
To: dev@directmemory.apache.org
Subject: Re: MapDB
References: <5099AE92.5060808@gmail.com>
 <CAAnVOG2Gz0T4Gew=CUwZ+0bWd_mF4P50TD-whAjw9mroJHfDhA@mail.gmail.com>
 <CAN9CcB_rnDsKWA-yFwvmrSS7ZO36wG4JF2K4BSKAkZAaghWvGg@mail.gmail.com>
In-Reply-To: 
 <CAN9CcB_rnDsKWA-yFwvmrSS7ZO36wG4JF2K4BSKAkZAaghWvGg@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

you are right.
Performance difference between inMemory and onDisk store is only 50% if 
data fits into memory.
MapDB does not use page buffer like most databases. It leaves page 
caching on OS.

instead MapDB has instance cache for already deserialized objects.

Jan


On 10/11/12 15:36, Julien Vermillard wrote:
> Hi, just a small comment.
> As mapDB use memory mapped files (mmap) using it for offloading memory to
> disk is probably not a performing idea since the O/S will already cache the
> file in memory the file for you :
> https://www.varnish-cache.org/trac/wiki/ArchitectNotes
>
> You will basicly have two memoery cache and the O/S will beat the
> directmemory one.
>
> My 2 cents,
>
> Julien
>
> On Wed, Nov 7, 2012 at 8:54 AM, Raffaele P. Guidi <
> raffaele.p.guidi@gmail.com> wrote:
>
>> Hi and thanks for your interest and offer! I know and appreciate your
>> project and made plans to integrate it as disk store since the early days.
>> We sure need good serialization libraries, in fact, in addition to provide
>> support for many different ones (kryo, msgpack, protostuff, etc) we
>> recently welcomed lightning (an unsafe based serializer by Christophe
>> Engelbert) as a subproject to complement our codebase with a fast (and
>> in-house) choice.
>>
>> I think we have more than one reason to integrate and share code:
>>
>>
>>     1. DirectMemory could make good use of mapdb to serialize least
>>     frequently used items to disk and free memory
>>     2. DirectMemory could implement a MapDB disk based store in addition to
>>     the bytebuffer and unsafe ones
>>     3. MapDB could take advantage of DM's componentization approach to
>>     support multiple serializers (we believe each one has its advantages in
>>     different scenarios)
>>     4. MapDB could use DM to write items to an off-heap before writing to
>>     disk (asynchronously) to improve speed
>>     5. We could merge our serialization efforts (I believe lightning is very
>>     fast and worth to be considered) and provide an even better solution or
>> two
>>     alternative implementations
>>
>> In both cases we would be open to contribution in different forms - just
>> contributing patches or with you to join us and the ASF as module or
>> subproject (the latter options have to undergo a formal vote by all project
>> members, of course) as I strongly believe that merging efforts would bring
>> to a better and more complete product.
>>
>> What do you think about it?
>>
>> Regards,
>>      Raffaele
>>