directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Alderson <Martin.Alder...@salfordsoftware.co.uk>
Subject RE: [ApacheDS] [JDBM] Index .db files grow from attribute replace
Date Mon, 30 Mar 2009 12:26:24 GMT
I've found the problem with this now.

It seems to be a bug within JDBM 1.0.  This was fixed in JDBMs CVS repository back in February
last year - before we branched: http://jdbm.cvs.sourceforge.net/viewvc/jdbm/jdbm/src/main/jdbm/btree/BTree.java?r1=1.17&r2=1.18

I guess we just branched the 1.0 code.  Can we just go ahead and apply this patch or should
we sync up with the latest JDBM trunk?

Thanks,

Martin



-----Original Message-----
From: Martin Alderson [mailto:Martin.Alderson@salfordsoftware.co.uk] 
Sent: Thursday 19 March 2009 18:07
To: dev@directory.apache.org
Subject: [ApacheDS] [JDBM] Index .db files grow from attribute replace

Hi all,

Firstly, apologies if this mail arrives in HTML format.  My company is in the middle of switching
e-mail systems and may be converting all outgoing mail to HTML at the moment.

I have found that when just replacing an indexed attributes value the JDBM table seems to
grow.  From doing this 300000 times I find that I have a 66 meg uid.db file which is growing
at the rate of around 2.2 meg per 10000 modifications and showing no signs of slowing down.

This is with a stock ApacheDS 1.5.4.  The following code should allow you to reproduce the
problem:

////////////////////////////////////////////////////////////
import java.util.Hashtable;
import javax.naming.Context;
import javax.naming.directory.BasicAttribute;
import javax.naming.directory.DirContext;
import javax.naming.directory.ModificationItem;
import javax.naming.ldap.InitialLdapContext;
import javax.naming.ldap.LdapContext;

public class StressTestIndex {

    public static void main (String[] args) throws Exception {
        Hashtable env = new Hashtable();
        env.put(Context.SECURITY_PRINCIPAL, "uid=admin,ou=system");
        env.put(Context.SECURITY_CREDENTIALS, "secret");
        env.put(Context.SECURITY_AUTHENTICATION, "simple");
        env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
        env.put(Context.PROVIDER_URL, "ldap://localhost:10389/");

        LdapContext ldapContext = (LdapContext) new InitialLdapContext(env, null);
        
        for (int i = 0; i < 100000; i++) {
            ldapContext.modifyAttributes("uid=admin,ou=system", new ModificationItem[] { new
ModificationItem(DirContext.REPLACE_ATTRIBUTE, new BasicAttribute("uid", "admin")) });
        }
    }

}
////////////////////////////////////////////////////////////

Emmanuel pointed me to the nifty LaunchDiagnosticUI extended request which I send as:
  ldapContext.extendedOperation(new LaunchDiagnosticUiRequest(0));

Looking at the JDBM table with this tool it looks like the index only has the one entry in
it as it should - so I think the rest of the space in the db file must be wasted space.  Does
anyone know if the JDBM table can be compacted?  If it can should ApacheDS be doing that periodically?

Thanks,

Martin



Mime
View raw message