LDAP Renaissance

Page edited by Alex Karasulu

=20

Architecting the Modern LDAP Renais= sance:
The Apache Directory Vision

by Alex Karasulu, Founder of the Apache Directory Project, October 2002
This paper was later, in September 2007, submitted to and accepted by the <= a href=3D"http://www.guug.de/veranstaltungen/ldapcon2007/" title=3D"www.guu= g.de" rel=3D"nofollow">1st International Conference on LDAP, Cologne (G= ermany)

Abstract

Directory technology is an indivisible cornerstone in computing science = and LDAP specifically is essential in several industries however it is seve= rely underutilized. One would expect the demand for LDAP to increase as inf= rastructures and the Internet grow, with more boundaries, nodes, services, = users and ways of doing business emerging rapidly. Directories inherently s= olve integration problems yet more integration problems appear while comple= xities of existing problems are compounded. We're not witnessing a proporti= onal increase in the adoption rate of directories; namely LDAP directories.= This is not a coincidence. It is the result of a lack of several factors: = tooling support, courses on directory technology in academia, qualified dom= ain experts and rich integration constructs. More specifically, information= architects and key decision makers incorrectly choose to apply ad hoc solu= tions to problems rather than opting to using directories. If these limitat= ions are removed then there would be greater comfort, flexibility and adopt= ion. LDAP could do more than it does today namely in the area of provisioni= ng and workflow. LDAP could potentially experience a Renaissance with renew= ed interest due to increased demand to solve the classical integration prob= lems it was designed for and beyond. We discuss these limitations, and the = proposed means to remove them, all in an effort to express our vision at th= e Apache Directory Project. Our aim is clear; we intend to influence and in= cite other directory implementers and projects by our example to trigger wh= at we envision as the Modern LDAP Renaissance.

= Drivers Leading to the Birth of the Directory

Several systems slowly appeared in the arsenal of tools used daily by th= ose within and across organizations. Data architects quickly could characte= rized two logical categories of information spanning across all domains:

Static information shared across applications and systems read frequ= ently yet seldom altered.
Information specific to an event within an application (a transactio= n) read infrequently however written rapidly in volume (data seldom accesse= d by external systems).

The Directory relates to this first type of data above which has the fol= lowing general properties:

Accessed by many clients
Partitioned across different locations in different organizations ow= ned by separate authorities
Structure data elements with data types (syntaxes) with semantics fo= r matching
Data easily fits into hierarchies
Highly cross referenced

The telecommunication industry in Europe, in particular, felt this burde= n while managing shared information, specifically subscriber directories. T= elephone companies in Europe, some national providers and other internation= al providers, managed profiles for subscribers, a subset of which, needed t= o be published for public access.

The ITU responded with the X.500 series of specifications. The Directory= specifications addressed these specific concerns for accessing information= shared across organizations, located in separate stores and managed by sev= eral different authorities. The access model and authoritative model enable= d organizations to publish subscriber information and allow others to acces= s it while delegating the management of access control, schema and other as= pects to the proper authorities. As a standard it was way ahead of its time= .

The X.500 specifications reveal a unique data access and authoritative m= odel designed specifically for the category of static shared data we define= d above. It is a cornerstone technology as is the relational database for t= he second category.

Demand= for Directories Should Be High

Directories expose a unique model for efficiently accessing relatively s= tatic information shared across multiple systems. Directory protocol design= ers intended the protocol to address a class of specific problems through a= standard means to access shared information centrally while providing high= availability. Existing storage paradigms failed to solve these problems du= e to inherent limitations in their design.

These data management problems fall under the domain of data integration= . To integrate systems one must integrate both the data and the processes. = Interoperating systems often need to access the same information however ma= ny systems simply duplicate that information (in relational databases) rath= er than accessing it from a centralized service (the directory). As a resul= t multiple copies of the same information exist across these systems. Data = synchronization technologies emerged to address these issues of duplication= such as meta-directories (not real directories). These technologies never = really solved all the issues of data duplication: they merely facilitated t= he export, transformation and import of data across systems. The serious is= sue of determining the authoritative copy still remained.

For some time, commercial entities avoided addressing these integration = problems by centralizing access to this shared information. The market fuel= ed the push of OLTP systems since the transaction equated to money. As mark= ets became saturated as several competitors appeared, businesses started fo= cusing on gains through efficiency. To streamline development and maintenan= ce better integration was needed while allowing transactions to span across= interoperating systems. This increased the need for directories.

Aspects of growth naturally contribute to the demand for Directory techn= ologies. As the number of users, organizations, systems, services and machi= nes expand the problems of integration only increase without applying the c= orrect remedy. In fact, the vast majority of today's problems are primary i= ntegration problems. Hence one would expect the utilization of Directories = to increase proportionally with the increased demand. Unfortunately, a prop= ortional increase in demand for Directories has not been observed.

Barriers of Adoption=

Although Directories, namely LDAP Directories, have proliferated over th= e past two decades, their adoption has not been commensurate with the deman= d one would expect. Several factors may be attributed to this less than opt= imal adoption rate.

Directories still feel alien to most developers. Many admit, in theory, = to a directory as the ideal access model for shared information in their ap= plications however most fall back to using a relational database instead. W= hy?

First there is no formal education around directory technologies while s= everal courses around relational database systems exist. This might explain= in part why developers are much more comfortable with keeping their Direct= ory information stored and accessed from an RDBMS. As a result, finding qua= lified Directory experts is difficult. Every generalist knows how to use an= RDBMS, yet costly specialists are needed to design, operate and maintain d= irectories. The cost and availability of specialized engineers factor into = design decisions especially if the process is formally conducted.

Developers may also fall back to using an RDBMS instead of a Directory b= ecause of a lack of rich integration tier constructs (like triggers, views = and stored procedures). These very useful constructs are missing in Directo= ries yet have been available in relational databases for decades. Today's h= eavy integration needs demand these constructs. For example the demand for = the event driven provisioning of information could be elegantly solved usin= g triggers and stored procedures as entries are added, updated and deleted.= Data architects and developers in this respect have been left in the dark-= ages with LDAP while the RDBMS has given them what they wanted/needed at th= e price of compounding their problems with data duplication.

Perhaps the most significant factor driving these faulty decisions is th= e lack of LDAP tooling. There are myriads of RDBMS tools for various aspect= s: tuning, accessing, and designing. Tools for the RDBMS appeared with a ve= ngeance to enable designers to rapidly prototype new OLTP systems and thus = bring them to market faster. Less confident generalists could even design d= atabases and manipulate them thanks to these tools which abstracted away th= e details to just get the job done. The tools made engineers and non-engine= ers productive in building database driven applications rapidly. Unfortunat= ely, there barely are any good browsers for LDAP much less topology/schema,= design and tuning tools. The barren tooling landscape leaves developers an= d data architects exposed enough to turn back to the RDBMS for immediate co= mfort. This temporary gain is at the cost of long term data bottle necks or= synchronization issues.

Regardless of which factor prevents most the uptake of LDAP, the fact re= mains: the wrong decisions are being made more often than the right ones. I= t's a matter of time before Directories are considered antiquated technolog= ies unless they can accommodate modern needs. Directories must be compellin= g enough for data architects and developers to decide to utilize them for t= heir strengths in theory and in practice.

Renovating (LDAP) Directories for the 21st Century

Several factors are not under the immediate control of the LDAP communit= y however enough are to bring LDAP to the 21st century. First and foremost,= LDAP tooling must improve. We need tools to model LDAP directories properl= y for topology and schema. Designers should be guided on how they can prope= rly use more of the esoteric constructs to control the naming, and hierarch= y of the directory. There's more to directory design than just objectClasse= s and attributeTypes.

Secondly the protocol must expand LDAP to include standard mechanisms fo= r declaring triggers, stored procedures and views. These two aspects alone = contribute to the vast majority of reasons why relational databases store i= nformation best accessed from a directory. Allowing these new constructs fo= r a directory will enable virtualization, provisioning, referential integri= ty, and server interaction with the external environment. Rather than only = exposing access to static information (white pages) the directory could be = utilized to solve many more problems confronting designers and developers i= n today's integration era. The Directory would truly become the Swiss Army = Knife(TM) of integration tools in the data architect's tool chest.

Conclusion: The Aim of the Apache Directory Project

At the Apache Directory Project, there are two LDAP specific projects. T= he first is a pure Java LDAP Server called Apache Directory Server (ApacheD= S) which has been written and certified by the Open Group for LDAPv3. It wa= s started in reaction to the often brittle code that was too hard to manage= : the code was written in C and had preprocessor directives strewn all over= it for porting.

The other LDAP related project is called Apache Directory Studio. It is = intended to provide the missing LDAP tooling support in addition to Apache = Directory Server specific management utilities.

The general motives for forming the Apache Directory Project can be summ= arized with the points below:

Avoid low level compiled language for: =09
- Reducing complexity, and increase productivity
- Enabling portability to other platforms with minimal effort
- Attracting more contributors with most common language (Java) =09
=09
Devise a more flexible dynamic server designed for: =09
- Extension
- Protocol experimentation
- Hot plug-ability
- Simplicity
=09
Experiment with and introduce new integration tier constructs for di= rectories: =09
- Triggers
- Views
- Stored Procedures
- Queues
=09
Provide the missing tooling critical for: =09
- LDAP Adoption (general for all LDAP servers)
- Increasing user comfort with LDAP
- Lessening the learning curve for those using LDAP
=09
Stimulate commercial vendors to adopt these new capabilities to rema= in competitive.

One of our aims is to induce thought in the LDAP community while stimula= ting its growth through simple useful products. The more satisfied LDAP use= rs are, the bigger the community using and interested in LDAP. This is wher= e simple intuitive tooling and a server come in hand. More than that, these= projects serve as an experimentation platform for devising new constructs = in the protocol and evaluating their utility without breaking with the exis= ting protocol. This is why certification is very important to us as we devi= se new constructs to make LDAP much more useful to its users.

Ultimately we want to see changes appearing in the protocol standard whi= ch make LDAP a more familiar hence appealing, and easier to use technology.= With standards around these new constructs interoperability could be achie= ved. Users will not be bound to a specific server. We want to demonstrate t= he viability and profound effects of these new constructs in the protocol t= hrough our example while stimulating commercial vendors to do the same to r= emain competitive. It does not matter to us which LDAP server is used so lo= ng as the users' situation improves with these new productive protocol feat= ures to make LDAP serve its intended purpose and go beyond. We strive, in t= his way, to increase LDAP awareness, comfort and adoption to bring forth wh= at we call the Modern LDAP Renaissance.

Change Notification Preferences

View Online | View Chang= e