directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <elecha...@gmail.com>
Subject Re: JDBM version & performances
Date Fri, 18 Jan 2013 09:32:50 GMT
Le 1/18/13 10:24 AM, Kiran Ayyagari a écrit :
> On Fri, Jan 18, 2013 at 2:24 PM, Emmanuel Lécharny <elecharny@gmail.com>wrote:
>
>> Hi guys,
>>
>> yesterday, while working on the issue we have when the server is
>> brutally shutdown, I noticed that we are using a version fo JDBM which
>> contains some code taht we don't need anymore (the Snapshot code). I
>> also did some experiment using the TransactionManager, with various
>> configuration. Here are some findings :
>>
>> 1) configuration impact
>>
>> When we use the TM, we need to commit() all the modifications and to
>> sync the transactions peridodically. I tested many diffeent parameters,
>> like calling commit() for every modification, or once for 2
>> modifications etc (1, 2, 5,10, 20, 50, 100, 200, 500, 1000...). The very
>> same for the sync method with the same values (except that I always
>> start using the same number than the commit - it makes no sense to sync
>> more often that we commit - ).
>>
>> The result is that with 200 modifications per commit, and 500 mods per
>> sync, I get a decent performance (less than 5% slower than not using a
>> TM at all). I think it's a good compromise as we don't have many pending
>> modification in case we have a crash.
>>
>> There is one more parameter I haven't played with, the max number of mod
>> in a transaction (it should be >= nb mod per commit)
>>
>> Now, we have to leverage those parameters, which means we should call
>> the commits() and sync() only when the syncWorker is kicked.

>> it would be nice to do the sync/commit when server is idle(or less loaded)
> but there are less pending mods than the above threshold

There is one slight issue with the differed commit/sync : I'm not sure
it's a thread afe operation. IMO, it should be conducted either while
processing a modification (ie, counting the per btree modification), or
using a lock mechanism forbidding any modification to start when he sycn
is running.

To be double chcked.
>
>> 2) JDBM2 contains some extra code, as it's supposed to be a inmemory
>> MVCC BTree. We don't use this feature, but still, there is some extra
>> code called. I have conducted some performances on he old JDBM and teh
>> new one, and I was able to see that the old JDBM is 8 to 10 % faster
>> than the new one.
>>
>> this is good
>> Considering that we don't need the Snapshot feature offered by JDBM2, I
>> think we should switch back to the old JDBM.
>>
>>  how do we differentiate these 'old' and 'new'?

I'm updating the jdbm parent's project to reflect teh various versions.
The 'pld' one will be labelled jdbm1. The new one will be jdbm2.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com 


Mime
View raw message