directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre-Arnaud Marcelot ...@marcelot.net>
Subject Re: JDBM version & performances
Date Fri, 18 Jan 2013 11:21:52 GMT

On 18 janv. 2013, at 10:32, Emmanuel Lécharny <elecharny@gmail.com> wrote:

> Le 1/18/13 10:24 AM, Kiran Ayyagari a écrit :
>> On Fri, Jan 18, 2013 at 2:24 PM, Emmanuel Lécharny <elecharny@gmail.com>wrote:
>> 
>>> Hi guys,
>>> 
>>> yesterday, while working on the issue we have when the server is
>>> brutally shutdown, I noticed that we are using a version fo JDBM which
>>> contains some code taht we don't need anymore (the Snapshot code). I
>>> also did some experiment using the TransactionManager, with various
>>> configuration. Here are some findings :
>>> 
>>> 1) configuration impact
>>> 
>>> When we use the TM, we need to commit() all the modifications and to
>>> sync the transactions peridodically. I tested many diffeent parameters,
>>> like calling commit() for every modification, or once for 2
>>> modifications etc (1, 2, 5,10, 20, 50, 100, 200, 500, 1000...). The very
>>> same for the sync method with the same values (except that I always
>>> start using the same number than the commit - it makes no sense to sync
>>> more often that we commit - ).
>>> 
>>> The result is that with 200 modifications per commit, and 500 mods per
>>> sync, I get a decent performance (less than 5% slower than not using a
>>> TM at all). I think it's a good compromise as we don't have many pending
>>> modification in case we have a crash.
>>> 
>>> There is one more parameter I haven't played with, the max number of mod
>>> in a transaction (it should be >= nb mod per commit)
>>> 
>>> Now, we have to leverage those parameters, which means we should call
>>> the commits() and sync() only when the syncWorker is kicked.
> 
>>> it would be nice to do the sync/commit when server is idle(or less loaded)
>> but there are less pending mods than the above threshold
> 
> There is one slight issue with the differed commit/sync : I'm not sure
> it's a thread afe operation. IMO, it should be conducted either while
> processing a modification (ie, counting the per btree modification), or
> using a lock mechanism forbidding any modification to start when he sycn
> is running.
> 
> To be double chcked.
>> 
>>> 2) JDBM2 contains some extra code, as it's supposed to be a inmemory
>>> MVCC BTree. We don't use this feature, but still, there is some extra
>>> code called. I have conducted some performances on he old JDBM and teh
>>> new one, and I was able to see that the old JDBM is 8 to 10 % faster
>>> than the new one.
>>> 
>>> this is good
>>> Considering that we don't need the Snapshot feature offered by JDBM2, I
>>> think we should switch back to the old JDBM.
>>> 
>>> how do we differentiate these 'old' and 'new'?
> 
> I'm updating the jdbm parent's project to reflect teh various versions.
> The 'pld' one will be labelled jdbm1. The new one will be jdbm2.

Everything sounds good to me.

Regards,
Pierre-Arnaud


> -- 
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com 
> 


Mime
View raw message