Hi Ole, Stefan,

I just wanted to add some more information to this.  LDAP schema according to the standard can
be composed of the following kinds of entities which govern various aspects of the schema
controlling entries:

* Syntaxes
* MatchingRules
* AttributeTypes
* ObjectClasses
* MatchingRuleUses
* DitStructureRules
* DitContentRules
* NameForms

In ApacheDS 1.5.0 and above we have some additional schema entities specific to ApacheDS which
were devised to facilitate the dynamic extension of the schema:

* Comparators
* Normalizers
* SyntaxCheckers

These atomic elements allow users to actually implement and load code into the server to create
new Syntaxes and MatchingRules.   They are the building blocks for defining the behavior of Syntaxes
and MatchingRules.

In traditional LDAP servers which lack these elements there was no way to dynamically add new
matchngRules or syntaxes without a restart if you could at all extend the server to do so.  Basically
you were stuck with the syntaxes and matchingRules hard coded into these servers.  The protocol
standards which define a grammar for specifying matchingRules and syntaxes are extremely simple.
They merely state that a matchingRule or syntax exists within the server with an OID for it.  This is
not enough to specify the behavior for these constructs.  For example a syntax spec might look like
so:

( 1.3.6.1.4.1.1466.115.121.1.15 DESC 'Directory String' )

As you can see there is not much to this.  It is merely a declaration that the a syntax exists with an
OID called 'Directory String'.  There is no information provided in the specification on how to validate
this syntax.  This is why so many legacy servers hard code the logic to validate the common
syntaxes.  This also means extending the syntaxes supported by these servers is either impossible
or require writing validation code using some proprietary interface for it.  If at all possible this
requires a restart and downtime.

I created the concept of a syntaxChecker for ApacheDS which has a simple description that may
include code to perform the validation.  Here's what the description for a syntaxChecker looks like:

( 1.3.6.1.4.1.1466.115.121.1.1 FQCN org.apache.directory.shared.ldap.schema.syntax.ACIItemSyntaxChecker  )

or another form for it is ...

( 1.3.6.1.4.1.1466.115.121.1.23 FQCN org.foo.BarSyntaxChecker  BYTECODE 'abc2d7ae3453...' )

The first form references a class that is included in ApacheDS and implements the SyntaxChecker interface
which is pretty simple here:

public 
interface SyntaxChecker
{
String getSyntaxOid();
boolean isValidSyntax( Object value );
void assertSyntax( Object value ) throws NamingException;
}
As you can see all a syntaxChecker does is announce the Syntax it validates and has two methods
to do the validation.  Once does a boolean check and the other does the same check but throws an
exception on a violation.

So in the first form the OID is really the OID of the syntax the SyntaxChecker validates.  The FQCN
is needed to load the class and add it to a registry for syntaxCheckers.  The second form is very
special in that it adds bytecode.  The syntax checker's bytecode is loaded into the server and is
then loaded into the jvm using a special classloader that searches the schema partition for the
class.  Once loaded an instance is created and added to the registry.  This way we can add new
syntaxCheckers to ApacheDS on the fly without a restart.

Now after adding the syntaxChecker description to the server an administrator can add the syntax
description for the new syntax with the same OID.  ApacheDS will not allow the addition of a syntax
without the presence of a syntaxChecker with the same OID to validate that syntax.

Does this make sense?

I won't go into matchingRules here but the concepts are similar.  MatchingRules unlike a syntax
require the presence of a comparator and a normalizer for the OID of the new matchingRule to
add to the server.

In conclusion these ApacheDS specific constructs allow for the dynamic extension of the server
where schema is concerned without a restart.  Unlike other LDAP servers you can add new
matchingRules and syntaxes to the server on the fly and not worry about down time.  I think this
is a very powerful feature that differentiates ApacheDS over other legacy servers.   Note that
these same concepts of dynamic extension are applied to Triggers and Stored Procedures which
have now appeared in ApacheDS 1.5.

<ot-idea>
Hmmm me thinks to me self that it might be a good idea to have an LS plugin which can allow
users to write these extension classes and load them into the server.  Write a SyntaxChecker
and right click in the pkg browser to deploy it to a server (connection).  How cool would that be?
</ot-idea>

Eventually I think we're going to see LDAP evolve to the point where the specifications need to
support a language independent means to extend the schema.  I have a feeling ApacheDS will
drive this revolution to reform this dilapidated protocol.  After all this was my original intent for
starting the Directory effort here.  I think a handful of crazy yet talented people in a community
can achieve this.

Alex

On 4/20/07, Ole Ersoy <ole.ersoy@gmail.com> wrote:
Hi Stefan,

It looks really good.  I'll probably we frequenting that when I need
reminders.  :-)

Also, I found Emmanuel's tip for browsing the Schema partition
(by adding an ou=schema connection) to LDAP studio really handy.

I'm not done reading yet, but I started with this:

=======================================================================
The schema of an LDAP server is comprised of object classes, attributes,
syntaxes and matching rules. Basically it defines which entries are
allowed within the server and how the server should handle them. In
contrast to the 1.0 release, ApacheDS 1.5.0 comes with a completely
redesigned schema subsystem. It allows to dynamically update the schema,
for instance it is possible to create new attribute types or object
classes on the fly, i.e. without restarting the server.
=======================================================================

Here's another attempt:

=======================================================================
The schema of an LDAP server is a set of entries of the following types:
- ObjectClass
- AttributeType,
- Syntax
- MatchingRule

The first three types of entries are used to the schema rules /
structure of other stored in the server.  ApacheDS 1.5.0 comes with a
completely redesigned schema subsystem. It enables dynamic schema
updates, like the creation of new AttributeType or ObjectClass entries
at runtime.
=======================================================================

I still need to figure out MatchingRule entries...:-)

I think the only thing that needed grammatic fixing was this part:

=======================================================================
It allows to dynamically update the schema,
=======================================================================

The rest is just a suggestion.

Cheers,
- Ole








Stefan Zoerner wrote:
> Hi all!
>
> I have started a little text for newbies on how to add custom elements
> to the schema with the help of the new schema subsystem of ApacheDS.
> First of all: I works fine to add elements with standard JNDI methods,
> well done, Alex!
>
> My first attempts for the text can be found here:
>
> http://cwiki.apache.org/confluence/display/DIRxSBOX/Add+your+first+elements+to+the+schema
>
>
> I plan to make this an introduction section to the schema topic in the
> Advanced User's Guide of ApacheDS 1.5, although there is obviously still
> some work to do. Any feedback on the current structure and state of the
> section is therefore highly appreciated! I am a newbie on this schema
> subsystem as well.
>
> The section is build around an example of adding a custom attribute type
> (numberOfGuns) and a custom object class (ship) to the schema in order
> to add entries like this one for instance:
>
> dn: cn=HMS Victory,ou=ships,o=sevenSeas
> objectClass: top
> objectClass: ship
> cn: HMS Victory
> numberOfGuns: 104
> description: a ship of the line of the Royal Navy, built between 1759
> and 1765
>
> For adding numberOfGuns and ship to the schema I have to use OIDs, but I
> think it is not worth to register them officially below
> 1.3.6.1.4.1.18060.0.
>
> Should I use obvious fun data like 9.9.9.9.9.1 and 9.9.9.9.9.2, describe
> the idea of OIDs and that a user should normally obtain a unique
> starting point?
>
> Or should I use something which starts with 1.3.6.1.4.1.18060.0 and add
> it on the list. Perhaps we can add a branch for documentation examples.
>
> What do you think?
> Greetings from Hamburg,
>     Stefan Zoerner (szoerner)
>
>
>