Mailing-List: contact dev-help@directmemory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@directmemory.apache.org
User-Agent: K-9 Mail for Android
In-Reply-To: 
 <CAAnVOG3Onbn7ejOiCZqw-Ax5kcngck6fhR+gTja7P3EAijaReg@mail.gmail.com>
References: <5099AE92.5060808@gmail.com>
 <CAAnVOG2Gz0T4Gew=CUwZ+0bWd_mF4P50TD-whAjw9mroJHfDhA@mail.gmail.com>
 <509A208D.3020609@gmail.com>
 <CAAnVOG3yq1p8DsgMgmTHdf51Cj9inhHZ+wyhy0XC0+zHDLCvRA@mail.gmail.com>
 <509B0F75.9000104@gmail.com>
 <CAAnVOG3Onbn7ejOiCZqw-Ax5kcngck6fhR+gTja7P3EAijaReg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="----6ZCIKMDMHHGVFWFF8CF2S86748HU58"
Subject: Re: MapDB
From: Christoph Engelbert <noctarius@apache.org>
Date: Thu, 08 Nov 2012 06:11:32 +0100
To: dev@directmemory.apache.org
Message-ID: <8a784379-e39c-44f5-9d5b-f02de5bf6e37@email.android.com>

------6ZCIKMDMHHGVFWFF8CF2S86748HU58
Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: 8bit

Unsafe allocator alone won't speed it up that much because I guess he relied on standard Serialization for his test. 


"Raffaele P. Guidi" <raffaele.p.guidi@gmail.com> schrieb:

>I don't believe integers or even 1kb buffers are the right case for
>directmemory. Small keys, large payloads (+4kb) are. In any case: how
>were
>the heap and xxmaxdirectmemory settings? And did you try the Unsafe
>storage
>as well? It would be helpful if you could share the benchmark code.
>
>Thanks,
>    R
>Il giorno 08/nov/2012 02:49, "Jan Kotek" <kjan80@gmail.com> ha scritto:
>
>> Hi,
>>
>> MapDB now has DirectMemory storage.
>> I made little test to compare our two libraries + java.util.TreeMap
>>
>> All tests are on 64bit Linux, 16GB RAM, JDK6. Default JVM settings.
>> Source code is here: https://github.com/jankotek/**
>>
>mapdb-junk/tree/master/src/**main/java/**directMemoryBenchmark<https://github.com/jankotek/mapdb-junk/tree/master/src/main/java/directMemoryBenchmark>
>>
>>
>> First is performance test. How long does it take to insert and fetch
>1
>> million records, each 1KB large.
>>  - DM - 6901 ms
>>  - MapDB - 2045 ms
>>  - java.util.TreeMap - 940 ms
>>
>> Second is overhead test. It inserts pair of integers in cycle, until
>JVM
>> runs out of memory.
>> It should measure how much memory library each consumes for its own
>> structures.
>>  - DM - inserts 9 000 000 records and then it becomes very slow.
>Profiler
>> shows high GC activity.
>>  - MapDB - works about 10 minutes and than it runs out of memory with
> 217
>> 000 000 records.
>>  - java.util.TreeMap - crashes after 99 000 000 records , it consumed
>> about 12GB heap.
>>
>>
>> I am not sure I configured DM correctly, JVM kept crashing when I
>> increased size too much.
>> I ended with this setting:
>>
>>        CacheService<Object, Object> m = new DirectMemory<Object,
>Object>()
>>                 .setNumberOfBuffers( 30 )
>>                 .setSize((int) 1e8)
>>                 .setInitialCapacity( 10000 )
>>                 .setConcurrencyLevel( 4 )
>>                 .setDisposalTime(1000000)
>>                 .newCacheService();
>>
>>
>> Hope this helps.
>>
>> Jan
>>
>>

-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.
------6ZCIKMDMHHGVFWFF8CF2S86748HU58--