directory-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lecharny <elecha...@gmail.com>
Subject Re: JDBC/Derby backend
Date Wed, 07 Feb 2007 23:29:15 GMT
Juergen Weber a écrit :

> On 2/7/07, Alex Karasulu <akarasulu@apache.org> wrote:
>
>>
>> >> Has anybody tried a Derby backend for ds?
>>
>> BTW I inquired about using Derby with Debrunner a few years ago at an
>> AC.  He basically stated that it would be a bad move since Derby would
>> be challenged to deal with hierarchies.
>
>
> Well, IBM's LDAP Server for z/OS does use DB2 as backend and it's very 
> fast.

Sure. I already saw IBM IDS + DB2 managing more than 70 000 000 entries 
without a pb.

<note>
I don't want to enter into a religious war here...
</note>

But "very fast" means almost nothing, if you can't compare it to 
something else. How "Very fast" is faster than "fast" or "slow" ?

What is important with using a RDBMS as a backend is not only 
performance. You have much more than that :
- public acceptance : Oracle, DB2, ... are almost everywhere. It would 
be bad to ignore this fact
- fault tolerance/reliability : do I have to add anything?
- transaction support. RDBMS support it natively
- Knowlegde : so many DBA, so few JDBM programmers...
etc...

<note>
End of the potential religious war. You are returning back to the 
agnostic zone :)
</note>

>
> Of course you have to map hierarchical data structures to tables but
> IBM showed that this is possible.

It is definitively possible. No doubt.

>
> This http://www.openldap.org/faq/data/cache/378.html OpenLdap FAQ
> entry more or less discourages from an SQL backend.
> But in this same FAQ entry Kurt Zeilenga gives a link to an IBM paper
> describing their database backend which quotes good performance (which
> matches our user experience with the z/OS LDAP server).
> http://www.research.ibm.com/journal/sj/392/shi.html

They are both interesting papers, but both a little bit outdated. 
Anyway, the rational for using something different than a RDBMS is 
pretty clear.

>
> A JDBC backend for DS could combine fast Java network and data
> structure handling with a fast Enterprise DBMS.
>
> For embedded use Derby should serve well.
>
> End of '05 I ran some transaction tests against Derby and got about 20
> TX/sec on my PC which seemed very fast to me.

FYI, as of mid-june 2006, we did some tests and compared ADS and 
OpenLdap on a simple desktop (one CPU, 3Mhz, 1 Gb). Open ldap was able 
to do up to 1500 search per second, and ADS went up to 900 search req/s.

Does it mean the number you obtained is slow ? No. My bet is that they 
will be perfectly OK for 99% of all the Ldap Server installed around the 
world. And that you may also use a more powerfull computer to obtain 
better perfs.

> As transactions are not very important for LDAP servers, only read
> performance would matter.

Very true. This is why your 20 TX/sec are not really meaningfull for a 
Ldap server. Without transaction, I won't be surprised if we get much 
better numbers.

> It would definitely be interesting to have numbers for Berkeley DB
> (which of course is not relational).

Well, I think it's close to the numbers we have found.

But I want to stress a point : those numbers mean nothing at all by 
themselves. The question is not to have the fastest LdapServer on earth, 
but much more to have a powerfull server, which can evolve, easy to 
manage, easy to install, reliable, scalable, easy to support...

Using a RDBMS as a backend is a must in this kind of scenario (IMHO).

Now, you have also to ponder this with other points :
- workforce to implement it : we are all volunteers, working on ADS out 
of day job, during evening, night and week-end
- users needs : we don't really have time to test and experiment all the 
possible solutions. Then it's all about choice. We do choice, every day, 
everytime. I hope that those choices are not the worst we could do, but 
we may be wrong from time to time :)
- knowledge : sometime you favor a technology against another one 
because of limited knowledge. (you might feel so confortable with a 
piece of software that you prefer to use it well than having a limited 
use of another component you don't really know)

Here we are. We really want to use a RDBMS system for many reasons. We 
didn't found the time to explore this solution up to now. We need other 
volunteers to help us :)

Just join the effort ! We will really appreciate new blood and new 
vision : Apache community is just all about that !

Emmanuel

>
> Juergen
>
PS: We already use Derby inside Apache DS (in the 1.5 version) : The 
Multi-Master replication mechanism use Derby as a repository.

Mime
View raw message