directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Karasulu" <>
Subject Re: [ApacheDS] Specifying application level subtrees?
Date Fri, 21 Sep 2007 20:54:56 GMT
Hi Marc,

On 9/21/07, Marc Boorshtein <> wrote:
> Alex,
> > > Thats why many virtual
> > > directories have dynamic group plugins to make dynamic groups work and
> > > act like static groups.
> >
> > Yes this is one such approach but it scares me in terms of the size of
> > entries that
> > will be returned.  You're going to be caching a lot in memory.  Consider
> a
> > dynamic
> > group with 1M members.  You now have to stuff your entry to be returned
> with
> > 1M
> > values for the [unique]member attribute.  If the average size for a
> member's
> > internationalized
> > DN is 512 bytes then we're going to return an entry which is 512M+ in
> size.
> > I don't want
> > to keep this entry in memory as the server builds it to send it out the
> > door.
> >
> If the application were pulling all members back then yes, this would
> be a major issue.  It is not however a requirement to bring all the
> members into the virtual directory and then return them to the client
> to test a group membership.  Generally a group test will look like:
> ldapsearch ... -b "cn=my group,ou=groups,dc=domain,dc=com" -s base
> "(uniqueMember=cn=me,ou=people,dc=domain,dc=com") 1.1

Very true but I was specifically talking about reading the entire group.
But you're
right this would work as well as a Compare request which is even cheaper to
on most servers.

> If the user is a member of the group then the group dn will be
> returned as the only entry.  While there are definitely really badly
> written applications that will retrieve the entire group and then do
> an evaluation thats pretty rare.

Yeah this is what I am afraid of.  I know of many apps like this
unfortunately.  Some
are really big commercial applications yuk!

Even if it does happen you would
> have the same problem with a static group.

Yep this is the reason why static groups are useless when the set increases
in size.

I've actualy deployed this
> solution with Oracle's virtual directory where the static members were
> 150k-500k members (and is a 256mb entry any better then a 512mb entry?
> :-D) and it worked very well.

Haa!  That's crazy. Try doing that under heavy load.  I'd like to see how
the VD then handles OOM

I'm still not sure what the advantage of your solution is over a
> dynamic group, but I'm probably missing something.

Hmmm are you up to date on the X.500 admin model?  SubtreeSpecifications and
what not?
The whole point is to leverage the same concepts rather then the standard
dynamic group
being based on the LDAP URL.  SS is much more flexible.

> I'd love to do that but for the reasons above I'm constrained.  The search
> > control based
> > approach is perhaps the best way to do this but then again the client
> has to
> > get a bit
> > smarter.  Perhaps both options can exist and when the size of the group
> is
> > beyond some
> > threshold the static group like lookup fails?
> >
> well, applications just aren't written well.  1/2 the commercial LDAP
> applications out there have poorly written LDAP modules that were
> built for a specific customer and were just 'released'.


> This get's me to another point.  LDAP needs a partial result return
> protocol
> > response
> > which could allow streaming large amounts of attributes back to clients
> to
> > prevent
> > excessive memory consumption.
> >
> > > The answer to 'why dont we do it' is that applications need to be able
> > > to consume it.
> >
> > Yep this is the biggest problem but applications need to change and so
> does
> > LDAP
> > otherwise we're stuck doing stupid things like HTTP did with
> statelessness
> > for decades.
> >
> Good luck to you there...I don't know that there will ever be an
> LDAPv4 but I suppose thats another discussion.

As with everything it's up to the community.  I don't know either and it's
probably a pipe dream
but oh what a good dream it is.

> Incidentally Authorization is already handled correctly by the X.500 model
> > which is being
> > specified in LDAP with the work being done by Steven Legg here:
> >
> >
> >
> > Authorization does not need this after this draft becomes an RFC which
> shows
> > a strong
> > liklihood.  ApacheDS uses this same model - this is why we stuck so
> close to
> > the X.500
> > standards in our AC implementation.
> >
> Well, I'm not a huge X.500 fan but you say tomatoe....I'm curios to
> check out the spec though.  Thanks for the pointer.

Well you're going to need to be.  I know reading X.500 specs make the best
ambien replacement in the world but LDAP is based on X.500 and is moving
closer to it.  IMO LDAP was too lightweight in an adverse reaction to the
weight of X.500.  So now people realize we have to embrace X.500 concepts
and in particular the admin model.  Lookie here ..

There's much more coming down the pike.  I've been dreaming about the day
this would happen.  The admin model is going take the ad hoc crap out of
implementations today.  Standards are the only way to reach true
across these ideas here.

> Getting back to groups.  This proposed mechanism for dynamic groups has
> it's
> > basis in X.500.
> > The excellent points regarding client consumption that you sited can be
> > ameliorated using some
> > existing protocol features such as extended operations and search
> controls
> > but we must explore
> > the impact of them on clients and on the server.  I smell a potential
> draft
> > for a new specification
> > here.  Interested?
> >
> Always glad to be involved and bounce ideas around.  I think I take a
> different tact but thats probably OK too.

Sure definitely.

Extended operations & controls are great and all but applications just
> don't use them.  If I were to design my super group it would have 4
> requirements:
> 1.  Members can be dynamic
> 2.  Members can be static
> 3.  Members can be other groups
> 4.  I can test a group membership with a single ldap search call

I like these items here especially #3 which did not occur to me until now.
Also note you can use a compare
op too but you're right most apps don't have a freakin clue about how to do
a compare either.   The name of
this game is to cater to the dumb clients already out there that won't

Now how to do this under the covers I bet you are far better at
> answering then me.  Again I can do this with a virtual directory now
> (MyVirtualDirectory has a DynamicGroups insert and a EmbeddedGroups
> insert that does this).  Maybe ApaceDS can do it better because it has
> all of its data locally (actually I'm positive it could).

We do have our class of problems but we'll solve them with time.  My whole
angle with this stuff
is to do it while extending (through IETF) and complying with standards and
expected semantics.
ADS is trying to achive these feature carefully without being another ad hoc
concoction to respond
to the market.  We want to outlast the fads.

In fact I am certain virtualization can have a solid theory behind it
through the X.500 admin model.
I'm in the process of defining conceptually views for LDAP using the admin
model and stored
procedures.  This however is future work.

I do say I'd be very interested to get the other open source ldap
> proejects (opends, openldap, penrose & fedorads) opinions on the
> matter.

Hmmm as you know I've been intricately involved with the Penrose peeps
having helped them use
apacheds under the hood.  I withdrew from them for various reasons but one
was because they
don't have a clue about the protocol and X.500.  Penrose is just an ad hoc
concoction to make money
quick. Nothing wrong with that but it bastardizes the technology with
one-offs jammed in to make
it to market.  You need a balance.

The other efforts are legitimate IMO but these folks are best engaged
through an open specification


View raw message