On Fri, Jan 18, 2013 at 2:24 PM, Emmanuel Lécharny <elecharny@gmail.com> wrote:
Hi guys,

yesterday, while working on the issue we have when the server is
brutally shutdown, I noticed that we are using a version fo JDBM which
contains some code taht we don't need anymore (the Snapshot code). I
also did some experiment using the TransactionManager, with various
configuration. Here are some findings :

1) configuration impact

When we use the TM, we need to commit() all the modifications and to
sync the transactions peridodically. I tested many diffeent parameters,
like calling commit() for every modification, or once for 2
modifications etc (1, 2, 5,10, 20, 50, 100, 200, 500, 1000...). The very
same for the sync method with the same values (except that I always
start using the same number than the commit - it makes no sense to sync
more often that we commit - ).

The result is that with 200 modifications per commit, and 500 mods per
sync, I get a decent performance (less than 5% slower than not using a
TM at all). I think it's a good compromise as we don't have many pending
modification in case we have a crash.

There is one more parameter I haven't played with, the max number of mod
in a transaction (it should be >= nb mod per commit)

Now, we have to leverage those parameters, which means we should call
the commits() and sync() only when the syncWorker is kicked.

it would be nice to do the sync/commit when server is idle(or less loaded) but there are less pending mods than the above threshold
2) JDBM2 contains some extra code, as it's supposed to be a inmemory
MVCC BTree. We don't use this feature, but still, there is some extra
code called. I have conducted some performances on he old JDBM and teh
new one, and I was able to see that the old JDBM is 8 to 10 % faster
than the new one.

this is good
Considering that we don't need the Snapshot feature offered by JDBM2, I
think we should switch back to the old JDBM.

 how do we differentiate these 'old' and 'new'?
I still have to check that we don't have any impact when running
concurrent modifications, but this is just a safety belt here : the way
we fixed the issue is orthogonal.

I'll keep ou inform today.

thank you for the details
Emmanuel Lécharny

Kiran Ayyagari