Hi Marc,

On 9/21/07, Marc Boorshtein <mboorshtein@gmail.com> wrote:
>It's essentially a means to group entries
> together and much more powerful
> than what is currently used in practice for dynamic groups: dynamic groups
> uses an LDAP URL to
> dynamically select the users for inclusion in the group.
>

Ok, I think I better understand what you are saying now.  I've seen a
lot of different group structures in directories.  From the simplest
static group to overly complex combinations of custom group entries
and custom az modules for applications.  The issue (or non issue
depending on how you look at it) is that a group is simply a directory
object.  What gives a group power is how the application deals with
the information.  The moral here being that if you have some alternate
idea of how to specify a group, i'm sure there are infinite ways of
specifying how to group users together but can the application
interpret that information?

Yep the app needs to either be aware or the server has to present the
dynamic group as a static group in structure.  Meaning inserting the
information into a static representation of the group on the applications
behalf.

Another avenue for it is to enable a membership extended operation to
ask simple questions like is this entry a member of a group.  As for the
following questions which return a set of entries:

i). list the groups an entry is a member of
ii). list the members of a group

I think a control on search would be best because you want to iterate through
it instead of caching a result set and returning it in a single response.  Obviously
this is due to the potentially massive result set you could get with some groups.
So it is best to stream it as you do with a db cursor on a result set. This is how
search works in ApacheDS - it pulls records from Partitions as clients ask for the
next one.  Hence the reason why a control would be best.

(case ii)
But the question remains how do you handle the semantics of such a search.
You might have a client that reads a dynamic group with the list members control
as an object scoped search request.  The client should then expect to see not
one result but several since it asked for all members with a specific control (client
can't be that ignorant).  So instead of getting a single result it handles several
results as if they executed the search for several entries.  This may be an OK route
to take.  Don't know for sure but perhaps we should talk about it.

(case i)
This one can be really easy. First let me revert and say ok we can just stuff all
the groups associated with the entry into a single entry result return and not stream
since the number of groups an entry is far far less than the number of members in
some groups.  So I'm doubling back here. 

This control should just insert a groups dynamic attribute into the returned entry
so the application that read the entry has this information when requesting it via
the control.  This makes sense for identity management and I think some servers
already have something like this in place.  SUN?  It makes sense to get all the
groups of a user for example when reading the user entry in one shot instead of
performing several searches to figure out this information which you may not be
able to do as the user you bind as.


For instance a dynamic group is much easier to manage in many ways
then a static group but applications don't generally know how to
compare a user against an ldap url.  

Exactly!

Thats why many virtual
directories have dynamic group plugins to make dynamic groups work and
act like static groups.  

Yes this is one such approach but it scares me in terms of the size of entries that
will be returned.  You're going to be caching a lot in memory.  Consider a dynamic
group with 1M members.  You now have to stuff your entry to be returned with 1M
values for the [unique]member attribute.  If the average size for a member's internationalized
DN is 512 bytes then we're going to return an entry which is 512M+ in size.  I don't want
to keep this entry in memory as the server builds it to send it out the door.

Ok this is a corner case but consider the count to be 10K members.  Still this is a 5MB
entry.  I am not sure I want to cache this either.  We make requests for membership information
quite frequently with large groups a lot of the server's memory is going to be sucked up to
handle lookups of groups.

So if you were to pair this group
specification idea with an interceptor that made it work like a static
group you might be on to something.

I'd love to do that but for the reasons above I'm constrained.  The search control based
approach is perhaps the best way to do this but then again the client has to get a bit
smarter.  Perhaps both options can exist and when the size of the group is beyond some
threshold the static group like lookup fails?

This get's me to another point.  LDAP needs a partial result return protocol response
which could allow streaming large amounts of attributes back to clients to prevent
excessive memory consumption.

The answer to 'why dont we do it' is that applications need to be able
to consume it.  

Yep this is the biggest problem but applications need to change and so does LDAP
otherwise we're stuck doing stupid things like HTTP did with statelessness for decades.

There is no az mechanism in a directory that can hide
the implementation.  Authorization is left to the applications that
are accessing the directory.

I'd like to approach this is as a general grouping mechanism discussion independent of
the application of groups.  Yes you can use it for Authorization but for now let's just
keep talking just about grouping.

Incidentally Authorization is already handled correctly by the X.500 model which is being
specified in LDAP with the work being done by Steven Legg here:

    http://tools.ietf.org/html/draft-legg-ldap-acm-bac-04

Authorization does not need this after this draft becomes an RFC which shows a strong
liklihood.  ApacheDS uses this same model - this is why we stuck so close to the X.500
standards in our AC implementation. 

Getting back to groups.  This proposed mechanism for dynamic groups has it's basis in X.500.
The excellent points regarding client consumption that you sited can be ameliorated using some
existing protocol features such as extended operations and search controls but we must explore
the impact of them on clients and on the server.  I smell a potential draft for a new specification
here.  Interested?

Alex