directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christoph Engelbert <noctar...@apache.org>
Subject Re: MapDB
Date Thu, 08 Nov 2012 05:11:32 GMT
Unsafe allocator alone won't speed it up that much because I guess he relied on standard Serialization
for his test. 



"Raffaele P. Guidi" <raffaele.p.guidi@gmail.com> schrieb:

>I don't believe integers or even 1kb buffers are the right case for
>directmemory. Small keys, large payloads (+4kb) are. In any case: how
>were
>the heap and xxmaxdirectmemory settings? And did you try the Unsafe
>storage
>as well? It would be helpful if you could share the benchmark code.
>
>Thanks,
>    R
>Il giorno 08/nov/2012 02:49, "Jan Kotek" <kjan80@gmail.com> ha scritto:
>
>> Hi,
>>
>> MapDB now has DirectMemory storage.
>> I made little test to compare our two libraries + java.util.TreeMap
>>
>> All tests are on 64bit Linux, 16GB RAM, JDK6. Default JVM settings.
>> Source code is here: https://github.com/jankotek/**
>>
>mapdb-junk/tree/master/src/**main/java/**directMemoryBenchmark<https://github.com/jankotek/mapdb-junk/tree/master/src/main/java/directMemoryBenchmark>
>>
>>
>> First is performance test. How long does it take to insert and fetch
>1
>> million records, each 1KB large.
>>  - DM - 6901 ms
>>  - MapDB - 2045 ms
>>  - java.util.TreeMap - 940 ms
>>
>> Second is overhead test. It inserts pair of integers in cycle, until
>JVM
>> runs out of memory.
>> It should measure how much memory library each consumes for its own
>> structures.
>>  - DM - inserts 9 000 000 records and then it becomes very slow.
>Profiler
>> shows high GC activity.
>>  - MapDB - works about 10 minutes and than it runs out of memory with
> 217
>> 000 000 records.
>>  - java.util.TreeMap - crashes after 99 000 000 records , it consumed
>> about 12GB heap.
>>
>>
>> I am not sure I configured DM correctly, JVM kept crashing when I
>> increased size too much.
>> I ended with this setting:
>>
>>        CacheService<Object, Object> m = new DirectMemory<Object,
>Object>()
>>                 .setNumberOfBuffers( 30 )
>>                 .setSize((int) 1e8)
>>                 .setInitialCapacity( 10000 )
>>                 .setConcurrencyLevel( 4 )
>>                 .setDisposalTime(1000000)
>>                 .newCacheService();
>>
>>
>> Hope this helps.
>>
>> Jan
>>
>>

-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message