directory-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kiran Ayyagari <kayyag...@apache.org>
Subject Re: Performance, 64Bit, IPv6, Reindexing
Date Fri, 21 Mar 2014 17:17:29 GMT
On Fri, Mar 21, 2014 at 10:30 PM, Emmanuel Lécharny <elecharny@gmail.com>wrote:

> Le 3/21/14 4:54 PM, Ernst Bech a écrit :
> > Hello,
> >
> > We am currently evaluating ApacheDS 2.0.0-M16. We come from using
> OpenLDAP
> > for a few years and are considering of moving to ApacheDS. The main
> > reasons for this is that we hope that ApacheDS has the following features
> > (I am fairly new to Java development and to this forum. So some of my
> > questions may be noobish)
> >
> > 1. 64 Bit runtime on windows
> Any JVM is 64 bits those days.
>
> > 2. IPv6 compatibility
>
> Java IO is fully IPV4/IPV6 capable.
>
> > 3. Reliable Master Master replication
>
> We do implement RFC 4533, as OpenLDAP does, so it should be reliable (of
> course, we might have some bugs, but we are trying to fix them as soon
> as we know about it).
>
> yeap, it is reliable and am using it in production servers, and yes
whenever we found
a bug we patched it asap

> >
> > And this with comparable performance.
>
> We don't claim to be as fast as OpenLDAP. A factor 2 should be
> considered as a good estimation when it comes to searches.
>
> >
> > To 1. I have installed ApacheDS on windows 7 (64 Bit) with Java 64 Bit
> > engine. With that It seems that ApacheDS is running 64 bit out of the
> box.
> > Or am I missing something?
> It's correct
>
> > To 2. I have made an connection with Apache Directory Studio using an
> IPv6
> > address and it worked, so I assume no issues here right?
> Right.
>
> > To 3. I cannot comment on Master Master replication as of yet.
> >
> > Regarding performance:
> >
> > While OpenLDAP was able to do an LDIF import of ~4000 entries in like 2
> > minutes
> Seems long.
>
> really long, looks like there must be something  odd with the env and/or
config

> > it took ApacheDS several hours.
> Then you have a problem.
>
> On my laptop, It would take 1min20 to load 4000 entries (and I still
> consider that it's slow).
>
> Hours ? What is your configuration, how many memory do you have, what
> kind of disk are you using, how many indexes do you have, and wht is the
> average entry size ?
>
> > Is it supposed to be like that or
> > are there ways to improve this?
>
> It's definitively not supposed to be like that. Hoever, loading entries
> in a started server will still be slow (count 50 entries per second).
>
> We are working hard to improve this :
> - the backend is entirely being rewritten, using a MVCC system
> - we have added a bulk-load tool that will not go through the whole
> server, but instead construct the btrees directly, making it it a order
> of magnitude faster (this feature will come with the new backend)
>
> > And while importing concurrent searches are really slow. It will not be
> an
> > option to switch to an "In Memory" backend because we need the data to be
> > persistent through reboots.
> Both requirement can be meet : with all your data in memory, the server
> should be fast, but they will also be persisted on disk. On a crash,
> yous hould be able to recover everything.
>
> The next backend version will be crash resistent.
>
> >
> > Re-indexing:
> >
> > I could not find information in the current documentation about how to do
> > this.
> This is not possible atm. The only way to do it is to extract the data,
> create the index, and reinject the data.
>
> a correction, building a new index is possible
1. configure a new index
2. restart the server (may take a while to start based on the number of
entries to be indexed)

> This is definitively a feature we wat to have in the server though.
>
>  it is available with the JDBM partition (which is the default backend for
any new partition, including ou=system and dc=example,dc=com partitions
that are configured by default)

> >  Assume I had imported a bunch of data into ApacheDS and then decide
> > to add an attribute to the index table. I found I can do that with Apache
> > Directory Studio by editing the configuration or alternatively by editing
> > directly in the ou=config tree, down to ou=indexes in the partition of
> > choice.
> > But how is the re-indexing done? Is it done automatically? Or by
> > restarting the server? Or do I need to kick it off somehow? I believe
> > there was a tool for this at an early stage of ApacheDS but not anymore.
> No, we never had a tool to do that... This will most certainly be added
> in the new backend.
>
> we used to have one during the pre 1.5.7v but it wasn't maintained
cause we merged this into the JDBM partition implementation

> Note that reindexing will require that the server is stopped and restarted.
>

Hope it helps...
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>


-- 
Kiran Ayyagari
http://keydap.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message