directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <>
Subject Re: Summer of Code Application
Date Mon, 13 Jun 2005 23:40:25 GMT
Enrique Rodriguez wrote:

> Hi,
> Regarding the Summer of Code, I'll add a few points on ApacheDS 
> projects and repeat what Trustin noted, for completeness.
Kudos to you Enrique for getting to this.  I was just about to try to 
summarize the candidate projects as promised to Mary in an email.  
Thanks!  Let me put some links to the repository where people can 
actually start looking at the code to get some traction.  Please look in 
line for links to svn ...

> 1)  DHCP - DHCP codecs work, but without broadcast support in MINA 
> and, fundamentally, in NIO, there isn't much to be done here until 
> NIO2 in (hopefully) JDK 1.6.  I stopped here, so the handler workflow 
> isn't totally complete, but should be straight-forward once MINA/NIO 
> supports broadcast.  I consider this code dead until then and not a 
> worthwhile project.

No common code yet!

> 2)  DNS - DNS codecs and workflow are in good shape.  As Trustin 
> mentioned, the handlers need to get wired into the ApacheDS backing 
> store.  Right now I have the handler stubbed out to simply echo the 
> DNS query as the response, but it is actually decoding and encoding.  
> Some time ago Alex added the DNS schema to the ApacheDS backend, so 
> that should be ready to go.  This is seriously only a couple days 
> work, but we put it on hold for a number of other initiatives and lack 
> of demand.  What's missing, but probably not worth a summer of work, 
> are the myriad of RFC's that add record types.  I coded in the most 
> widely used ones, the ones supported by the schema, such as A and MX, 
> but there are quite a few more.

No common code yet!

> 3)  Kerberos - Kerberos is also mostly working, and tested for interop 
> with Windows and Linux.  The major feature missing from RFC 1510 is 
> cross realm authentication, aka trust relationships.  I have this on 
> my plate already, and it is mostly backend work, figuring out now how 
> I want to lay this out in the DIT.  From more recent clarifications 
> documents, we are missing more modern encryption types, but these look 
> to be basic assembly based on crypto in the JDK or Bouncy Castle.
Protocol Provider (MINA):

Shared (Common Code):

> 4)  NTP - NTP is pretty much done.  It really just needs to be wired 
> into the ApacheDS server at some point.  One possible project here is 
> NTP authentication.  I didn't do any work on that, but the field is 
> supported by the codecs.  NTP has no configuration, so part of tying 
> it into ApacheDS would be to at least make the port configurable.
> So, same projects that would be good to do over the summer:

No common code yet!

> 5)  DNS GSS-TSIG - RFC 3645 - Generic Security Service Algorithm for 
> Secret Key Transaction Authentication for DNS (GSS-TSIG).  I would be 
> willing to mentor someone on this:
> 6)  Kerberos RC4-HMAC encryption type.  This would be a nice project 
> and would require next to zero messy integration work.  The package 
> directory/shared/kerberos/trunk/common/src/java/org/apache/kerberos/crypto 
> has the encryption engines, and you'd be writing a new one of these to 
> produce Kerberos Cipher Text based on the RC4-HMAC algorithm.
> There are also a couple "new" encryption types based on AES that we 
> don't support, but combined with this RC4-HMAC engine, would add up to 
> a nice-sized project.
> 7)  LDAP ACL - I know we want this but I'm no expert here, just 
> familiar from a usage standpoint with OpenLDAP.  In the past Alex has 
> mentioned RFC's 2820, 2829, 3112, and 3062.

See LDAP section below ...

> 8)  ApacheDS backing store - We don't talk about this much, but I 
> don't think many of us like the JDBM backing store.  We've talked 
> about doing versions with Derby or Prevayler.  JDBM has not had a 
> release since 2001 and doesn't appear to be in active development.  I 
> seem to recall also that the performance stats were poor.

Also we'd like to see an in memory store but I guess Prevayler covers 
this.  WRT Derby I've inquired with DeBruiner (sorry if I spelled the 
name wrong) at last ApCon about efficiency of hierarchical data storage 
and it did not look good ... this was one of the weaker points to Derby 

WRT to JDBM I agree its been lagging and is slow especially on writes 
but this is ok.  I think there are more efficient ways we can be using 
JDBM too however that needs more exploration.  Regardless we have yet 
another option since backends are pluggable: using SleepyCat's JE in 
place of JDBM.  A common interface for all BTree based stores can be 
used to switch between JE and JDBM.  However this may not be necessary 
since the plugability is achieved at the backend partition level.

> 9)  SASL - SASL has been discussed a bunch on this list.  I refer you 
> to the archives.  Specifically, we'd like to add GSS-API/Kerberos 
> support.


Generally for all ApacheDS backed services I recommend building out a 
configuration OU under the ou=system context/partition.  Services like 
Kerberos can use the Preferences API to manage their configuration 
information there on changes to defaults. It's like our own registry.   
Here's the code location for a Preferences implementation within the 
ApacheDS server core:

ASN.1 Libraries

1). Runtime

Currently Emmanuel is working in the sandbox to replace the existing 
ASN.1 libraries.  He has a solution that is 8x faster than the present 
implementation.  The present implementation implements a digester 
approach (yes like one used in commons for XML) but is based on matching 
patterns in the hierarchy of streaming tags: turns out to be a poor 
approach from a performance standpoint.  It uses rules to that push pop 
objects and operate on them.  It's dynamic and in theory thought to be 
more flexible.  It turned out to be a nightmare in terms of code 
management.  A finite state machine is much faster.

2). Stub compiler

Some theories and experiments but we're still deciding on the direction. 

3). More LDAP controls in Emmanuel's code and in present day library if 
the new code base will take more time.

People interested in ASN.1 (BER and DER encoding) and a stub compiler 
should connect with Emmanuel Lecharny and Alan Cabrera. 

LDAP Agenda, Direction, Road map etc ...

1). Subentries

In short we need to implement the subentries RFC to get to the next 
level where we can introduce Administrative Areas for various aspects: 
authorization, schema, collective attributes, replication etc..  
Administrative Areas are defined by subentries.  Kurt Zeilenga from 
OpenLDAP worked on the RFC on subentries which can be found here:

As you read into this various requirements will become apparent ... I 
have noted some here on the wiki but it is anemic to say the least:

I started out by adding code to parse and represent a refinement so they 
can be handled by the server in the shared code base since this might be 
used by clients as well later to interpret subentry information.  Here's 
a the new package location in SVN where I had started adding this 
stuff.  Not much is there so don't get excited :(.


* parser needs to be completed with these primitives. 
* add internal representation or data structure for quickly looking up 
an entries inclusion set within subtrees for evaluating different 
aspects of administration
* add subsystems using interceptors to handle aspects:
   - authorization
   - collective attributes
   - schema

2). More backend partition types as noted above by Enrique. 

BTW the Prevayler based in memory backend will save major disk IO down 
the line when used to replace the system backend partition.  All major 
subsystems that will be made more robust after subentries are 
implemented will most likely use the system partition to store 
persistent information.  Prevayler from my understanding puts an entire 
db into memory loading it on startup.   On shutdown changes are 
persisted or synch operations can be made intermittently. 

Right now to solve this problem I usually setup the system partition 
parameters for JDBM to use as much cache as needed.  This should result 
in theory to a similar behavior but I have not seen this to be the case.

3). Possible replacement for AspectJ. 

The JDK version issues with AspectJ have started to annoy me as they 
will others.  Perhaps we can remove the use of aspects all together 
since the point cuts are all static.  Don't know.  Perhaps we can use 
lower level libraries to achieve this effect with better efficiency.  
AspectJ runtime is not the fastest out there.

4). Support for LDAP Controls (implementation in server not in ASN.1 

  o Persistent search control
  o Sort control
  o Named References/ManageDsaIT

5). The core JNDI provider needs some more work. 

The provider lacks full functionality for referrals and other features.  
It would also be nice to be able to get namespace federation working so 
we can bridge the gap between the Naming code and the ApacheDS core 
code.  This way we can traverse namespaces and switch providers 
dynamically between namespaces.

6). Better transaction management and ACIDity instead of using buffer 
cache like mechanism with synch method calls

7). Triggers, views and stored procedures

Once subentries are in place I would like to have AAA's managing 
information for trigger, view and proc specifications.  Nothing like 
this is formalized yet in LDAP so its great material for potentially new 
IETF standards around these constructs.  The RDBMS world has em why not 

8). Correct Abandon operation handling

The Abandon request does not work in the server.  Low hanging fruit here 
but it needs to be done right since another thread has to be stopped.

9). Correct handling of time and size limits

The search request currently does not time out search requests that 
exceed these limits.  Search filters can be used to limit these for each 
entry requested since entries are streamed out of the server one at a 
time as they are requested by the JNDI.  Search does not load all 
candidate entries selected by a filter into memory then return them: the 
server would croak if we did that.

10). Complete support for most schema checking constructs

Everything is in place for schema checking but nothing exists today.  
The reason being I did not waste time on schema subsystem since it would 
need to be revamped once we implemented subentries.  Subentries for 
schema AAAs would store the schema info there rather than having one 
schema for the entire DIT mastered by a DSA. 

Basically an interceptor for schema management would use the subentry 
information to check for correct schema usage in different schema AAAs 
in the DIT.

11). Better error handling and easily understood messages

12). Logging

I personally did a poor job of this  To my credit much of the logging 
lines were added and removed while moving from one IoC container to the 
next during the Avalon days. 

13). Possible IoC Container Strategy

ApacheDS does not use a container and we have tried to be container 
independent to avoid introducing specificities. 

We want the server to be as embeddable as possible WRT the existing 
containers out there.  Hence we do not want say the Geronimo folks to 
have to introduce container specific code if they want to embed 
ApacheDS.  Plus we don't want protocol implementors and committers on 
the core to have to worry about learning a new container and its 
semantics.  Things are already hard. 

If we can keep the container code separated from the core then perhaps 
we can ship with a default container for the final executable service.  
Why try? We loose a lot not having a default container.  What do we loose:

Aspects like ...

 o management interfaces
 o logging
 o configuration
 o hot deployment of components
 o ...

Right now Enrique has attracted my attention to OSGi.  He makes 
excellent points about it.  It's the container of choice for embedding 
in appliances and other devices.  Plus it's got a written specification 
to it endorsed by major companies like IBM, Cisco and many others.  This 
is what makes it unique in comparison to the Geronimo GBeans, Spring and 
other Avalon, ComponentHaus productions.  All these containers are great 
btw so I'm not putting them down and we still reserve the right to 
integrate with them as well.  However OSGi seems like the best candidate 
to Enrique and I.


Perhaps (in a separate mail thread) you can elaborate a bit on OSGi 
advantages and the wrapper code you have introduced in the sandbox? 

14). Language Tags

I'd like to see language tags implemented for LDAP so pulling an 
attribute based on user profile locale can result in the correct value.  
For more info on language tags look here:

Yum another fresh new LDAP RFC.  Kudos to Zeilenga and friends at 
OpenLDAP.  So now the core JNDI provider can be used 
*java.naming.language property and search recall attributes based on a 

15). Support for collective attributes

K. Zeilenga is unstoppable: another great LDAP specification. 
Unfortunately this is also missing in ApacheDS:

Again we need subentries implemented first to do this correctly.

16). SASL/GSSAPI/Kerberos support

Nice to have some support for this.  Insecure LDAP is not LDAP according 
to the security requirements set forth in v3 specifications.  Related 
RFCs in LDAP land specifically include:


17). DNS-based service location


Comments thoughts with LDAP relationship?

LDAP: Summary

The LDAP space is huge and ripe for the pickings.  Directories represent 
a cornerstone technology so it is naturally vast and there are even more 
RFCs out there on LDAP.  I just hit upon perhaps a third to one half of 
them either directly or indirectly.  Step one is not to freak.  Step two 
is to start looking for simple areas where you can start working without 
dependencies.  As noted several things are dependent on subentry 
implementation but there are other independent efforts that can be 
concurrently developed as well.

In General: Summary

When looking at these lists try to isolate your interest if any and 
interface with us on the list to see how we can put your feet on the 
ground reading the existing code and formalizing an approach.  It's all 
about working together as a community to help one another see the best 
possibilities.  Meaning your efforts should not be in a vacuum. 
Otherwise you might be choosing implementation pathways leading to a 
dead end or incorrectly presuming things regarding the current operation 
of the server.  We can help you out but cannot read your mind: just your 

Ask questions! Post to the list and don't feel self conscious.  I know 
many people get intimidated posting to these lists.  No one will haze 
you for trying to be involved or if you don't have sufficient 
information.  We will try to point you to the proper resources if not 
explain things ourselves hopefully adding to those resources. 

Good luck,

View raw message