directory-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ernst Bech <Ernst.B...@caseris.de>
Subject Antwort: Re: Performance, 64Bit, IPv6, Reindexing
Date Mon, 24 Mar 2014 10:07:59 GMT
Hello,

thank you both for answering that fast.

1. Here are some details to the LDIF import data:
The LDIF file with 4000 entries I imported has a size of 20MB. So the 
average size is 5KB per entry. 2300 of 4000 entries are file content. All 
of the LDIF data is based on our own schema which is derived from person 
and from each other.
The hardware I'm doing the tests on has an Intel Core i7 CPU @ 2.8GHz 
processor and has 8 GB RAM. The hard drive is a 500GB Seagate ST3500418AS 
ATA disk and has 120GB free. The PC is 4 years old and I have Avira 
professional Security virus scanner running. For the tests I created 2 
extra indexes, one for sn and one for cn. For production use I will need 
alot more indexes, like 20.

2. To the reindexing:
So adding an index and restarting the server will do the reindexing 
automatically, that's fine :)
Does this also hold true, if I alter an index by changing the values of 
ads-enabled or ads-indexhasreserve?
Does this also hold true, if I delete an index?
Can you elaborate on the ads-indexhasreserve attribute? I don't know what 
it does and when I have to use it. It is set to TRUE on the 
ads-indexAttributeId=apacheRdn in the dc=example,dc=com of the default 
installation. But when I make my own partition it is set to FALSE.

3. Starting the server fails the first time:
Mostly after reboot or after I change something in the configuration the 
first time I try to start the server it starts and then ends itself. When 
I try again it stays started. Is that the expected behavior? Has it to do 
with bringing up the server to the new configuration?



Von:    Kiran Ayyagari <kayyagari@apache.org>
An:     "users@directory.apache.org" <users@directory.apache.org>, 
Datum:  21.03.2014 18:29
Betreff:        Re: Performance, 64Bit, IPv6, Reindexing



On Fri, Mar 21, 2014 at 10:30 PM, Emmanuel Lécharny 
<elecharny@gmail.com>wrote:

> Le 3/21/14 4:54 PM, Ernst Bech a écrit :
> > Hello,
> >
> > We am currently evaluating ApacheDS 2.0.0-M16. We come from using
> OpenLDAP
> > for a few years and are considering of moving to ApacheDS. The main
> > reasons for this is that we hope that ApacheDS has the following 
features
> > (I am fairly new to Java development and to this forum. So some of my
> > questions may be noobish)
> >
> > 1. 64 Bit runtime on windows
> Any JVM is 64 bits those days.
>
> > 2. IPv6 compatibility
>
> Java IO is fully IPV4/IPV6 capable.
>
> > 3. Reliable Master Master replication
>
> We do implement RFC 4533, as OpenLDAP does, so it should be reliable (of
> course, we might have some bugs, but we are trying to fix them as soon
> as we know about it).
>
> yeap, it is reliable and am using it in production servers, and yes
whenever we found
a bug we patched it asap

> >
> > And this with comparable performance.
>
> We don't claim to be as fast as OpenLDAP. A factor 2 should be
> considered as a good estimation when it comes to searches.
>
> >
> > To 1. I have installed ApacheDS on windows 7 (64 Bit) with Java 64 Bit
> > engine. With that It seems that ApacheDS is running 64 bit out of the
> box.
> > Or am I missing something?
> It's correct
>
> > To 2. I have made an connection with Apache Directory Studio using an
> IPv6
> > address and it worked, so I assume no issues here right?
> Right.
>
> > To 3. I cannot comment on Master Master replication as of yet.
> >
> > Regarding performance:
> >
> > While OpenLDAP was able to do an LDIF import of ~4000 entries in like 
2
> > minutes
> Seems long.
>
> really long, looks like there must be something  odd with the env and/or
config

> > it took ApacheDS several hours.
> Then you have a problem.
>
> On my laptop, It would take 1min20 to load 4000 entries (and I still
> consider that it's slow).
>
> Hours ? What is your configuration, how many memory do you have, what
> kind of disk are you using, how many indexes do you have, and wht is the
> average entry size ?
>
> > Is it supposed to be like that or
> > are there ways to improve this?
>
> It's definitively not supposed to be like that. Hoever, loading entries
> in a started server will still be slow (count 50 entries per second).
>
> We are working hard to improve this :
> - the backend is entirely being rewritten, using a MVCC system
> - we have added a bulk-load tool that will not go through the whole
> server, but instead construct the btrees directly, making it it a order
> of magnitude faster (this feature will come with the new backend)
>
> > And while importing concurrent searches are really slow. It will not 
be
> an
> > option to switch to an "In Memory" backend because we need the data to 
be
> > persistent through reboots.
> Both requirement can be meet : with all your data in memory, the server
> should be fast, but they will also be persisted on disk. On a crash,
> yous hould be able to recover everything.
>
> The next backend version will be crash resistent.
>
> >
> > Re-indexing:
> >
> > I could not find information in the current documentation about how to 
do
> > this.
> This is not possible atm. The only way to do it is to extract the data,
> create the index, and reinject the data.
>
> a correction, building a new index is possible
1. configure a new index
2. restart the server (may take a while to start based on the number of
entries to be indexed)

> This is definitively a feature we wat to have in the server though.
>
>  it is available with the JDBM partition (which is the default backend 
for
any new partition, including ou=system and dc=example,dc=com partitions
that are configured by default)

> >  Assume I had imported a bunch of data into ApacheDS and then decide
> > to add an attribute to the index table. I found I can do that with 
Apache
> > Directory Studio by editing the configuration or alternatively by 
editing
> > directly in the ou=config tree, down to ou=indexes in the partition of
> > choice.
> > But how is the re-indexing done? Is it done automatically? Or by
> > restarting the server? Or do I need to kick it off somehow? I believe
> > there was a tool for this at an early stage of ApacheDS but not 
anymore.
> No, we never had a tool to do that... This will most certainly be added
> in the new backend.
>
> we used to have one during the pre 1.5.7v but it wasn't maintained
cause we merged this into the JDBM partition implementation

> Note that reindexing will require that the server is stopped and 
restarted.
>

Hope it helps...
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>


-- 
Kiran Ayyagari
http://keydap.com


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message