directory-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <elecha...@gmail.com>
Subject Re: Performance, 64Bit, IPv6, Reindexing
Date Fri, 21 Mar 2014 17:00:33 GMT
Le 3/21/14 4:54 PM, Ernst Bech a écrit :
> Hello,
>
> We am currently evaluating ApacheDS 2.0.0-M16. We come from using OpenLDAP 
> for a few years and are considering of moving to ApacheDS. The main 
> reasons for this is that we hope that ApacheDS has the following features 
> (I am fairly new to Java development and to this forum. So some of my 
> questions may be noobish)
>
> 1. 64 Bit runtime on windows
Any JVM is 64 bits those days.

> 2. IPv6 compatibility

Java IO is fully IPV4/IPV6 capable.

> 3. Reliable Master Master replication

We do implement RFC 4533, as OpenLDAP does, so it should be reliable (of
course, we might have some bugs, but we are trying to fix them as soon
as we know about it).

>
> And this with comparable performance.

We don't claim to be as fast as OpenLDAP. A factor 2 should be
considered as a good estimation when it comes to searches.

>
> To 1. I have installed ApacheDS on windows 7 (64 Bit) with Java 64 Bit 
> engine. With that It seems that ApacheDS is running 64 bit out of the box. 
> Or am I missing something?
It's correct

> To 2. I have made an connection with Apache Directory Studio using an IPv6 
> address and it worked, so I assume no issues here right?
Right.

> To 3. I cannot comment on Master Master replication as of yet.
>
> Regarding performance:
>
> While OpenLDAP was able to do an LDIF import of ~4000 entries in like 2 
> minutes 
Seems long.

> it took ApacheDS several hours. 
Then you have a problem.

On my laptop, It would take 1min20 to load 4000 entries (and I still
consider that it's slow).

Hours ? What is your configuration, how many memory do you have, what
kind of disk are you using, how many indexes do you have, and wht is the
average entry size ?

> Is it supposed to be like that or 
> are there ways to improve this?

It's definitively not supposed to be like that. Hoever, loading entries
in a started server will still be slow (count 50 entries per second).

We are working hard to improve this :
- the backend is entirely being rewritten, using a MVCC system
- we have added a bulk-load tool that will not go through the whole
server, but instead construct the btrees directly, making it it a order
of magnitude faster (this feature will come with the new backend)

> And while importing concurrent searches are really slow. It will not be an 
> option to switch to an "In Memory" backend because we need the data to be 
> persistent through reboots.
Both requirement can be meet : with all your data in memory, the server
should be fast, but they will also be persisted on disk. On a crash,
yous hould be able to recover everything.

The next backend version will be crash resistent.

>
> Re-indexing:
>
> I could not find information in the current documentation about how to do 
> this.
This is not possible atm. The only way to do it is to extract the data,
create the index, and reinject the data.

This is definitively a feature we wat to have in the server though.

>  Assume I had imported a bunch of data into ApacheDS and then decide 
> to add an attribute to the index table. I found I can do that with Apache 
> Directory Studio by editing the configuration or alternatively by editing 
> directly in the ou=config tree, down to ou=indexes in the partition of 
> choice.
> But how is the re-indexing done? Is it done automatically? Or by 
> restarting the server? Or do I need to kick it off somehow? I believe 
> there was a tool for this at an early stage of ApacheDS but not anymore.
No, we never had a tool to do that... This will most certainly be added
in the new backend.

Note that reindexing will require that the server is stopped and restarted.

Hope it helps...


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com 


Mime
View raw message