directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emmanuel Lecharny" <elecha...@gmail.com>
Subject Re: Convo on IRC about schema
Date Thu, 07 Dec 2006 09:47:59 GMT
Hi Ersin,

sorry, but my buffer was also limited to 300 lines... I increased it to
3000, but I'm sorry to tell you that your interesting conribution has been
lost forever, except for us :(

Thanks anyway, it was quite an interesting convo, and it has helped me to
better understad some parts of the schema system.

And sorry if I was quite confusing, I had some strange misconception about
what we have to do, because of my lack of knolewdge in huge part of X500 :(

Emmanuel

On 12/6/06, Ersin Er <ersin.er@gmail.com> wrote:
>
> It would be better if your buffer did not cut the portion of the convo
> which I talked much more and told about much more useful stuff :-)
>
> On 12/6/06, Alex Karasulu <akarasulu@apache.org> wrote:
> >
> > Hi all,
> >
> > This was a convo about schema info on IRC.  My buffer does however cut
> > some of the conversation.
> >
> > Alex
> >
> >
> > <akarasulu> :)
> > <elecharny> not a big deal
> > <elecharny> ok
> > <akarasulu> even with OID you cannot have two different OID with same
> > names
> > <elecharny> so each partition has its own subset of the common schema
> > <akarasulu> you know this list of names each entity can have
> > <akarasulu> like cn and commonName for example
> > <elecharny> yeah
> > <elecharny> you can't reuse cn for something else
> > <elecharny> you should have only one cn in the world
> > <elecharny> like you have only one 2.5.4.5
> > <ersiner> Let me do X.500++ in my new thesis :)
> > <elecharny> yeah, good idea :)
> > <akarasulu> exactly
> > <elecharny> ok, so what were we talking about at the beginning ?
> > <ersiner> we came here from need of parsers
> > <elecharny> let say we have only one partition, cn=example,dc=com
> > <ersiner> alex was explaining that we would use them in two places
> > <elecharny> ahhh, yes, the parsers
> > <akarasulu> so we need those parsers
> > <elecharny> so having parsers will be used if a partition admin (or
> > somebody allowed to modify the partition schema)
> > <elecharny> want to add a new AT or OC
> > <akarasulu> for not only the description syntaxes defined by the
> > protocol but we're going to add new description syntaxes
> > <akarasulu> we're going to add one for each of the following :
> > <akarasulu> comparators
> > <akarasulu> normalizers
> > <akarasulu> syntaxCheckers
> > <akarasulu> these descriptions will have either a FQCN value in it (for
> > built in entities bundled with ADS) or a BYTECODE value (base 64
> > encoded) in it
> > <elecharny> the funny thing is that ifyou have to create a new SC,
> > <akarasulu> these will be used to store user added schema constructs
> > that need bytecode to be loaded from them
> > <elecharny> you will have to write a java class for it
> > <akarasulu> yeah this is for new SC
> > <akarasulu> yeah
> > <elecharny> and calssload it into ADS
> > <akarasulu> exactly
> > <akarasulu> well this mechanism can be used
> > <akarasulu> you could use this for extended OpenLDAP file format too btw
> >
> > <elecharny> and then, you may generate the schema data out of the class
> > :)
> > <elecharny> for instance, for a SC, all you need is :
> > <elecharny> - the SC OID,
> > <akarasulu> this way though we completely describe all the elements
> > needed for describing a new schema
> > <elecharny> and the SC description
> > <akarasulu> well not really
> > <akarasulu> elecharny listening?
> > <ersiner> guys i need to go offline for ph.d bullshit
> > <elecharny> yep
> > <akarasulu> np
> > <akarasulu> bye ersiner
> > <ersiner> cul8r
> > <akarasulu> elecharny let me clarify ...
> > <elecharny> we will keep this convo and post it to you ersiner
> > <ersiner> oh thanks
> > <ersiner> really.
> > <ersiner> bye.
> > <-- ersiner has quit ("Ex-Chat")
> > <akarasulu> We want to be able to dynamically load a newly added SC
> > without having to restart the server right?
> > <elecharny> yes
> > <akarasulu> someone adds a new schema
> > <akarasulu> they add normalizers and comparators and SCs with it
> > <elecharny> yes
> > <akarasulu> so we want it so this shit is loaded immediately into memory
> >
> > and a SC is instantiated and added to it's registry
> > <akarasulu> so it is made available
> > <elecharny> yes
> > <akarasulu> and can be seen via cn=schema at least
> > <elecharny> yes
> > <akarasulu> if not by other partition specific subentries
> > <akarasulu> so we need to invent a new description syntax for these
> > entities (SC, N, and C)
> > <akarasulu> for example ...
> > <akarasulu> ( 1.2.34.32.2.21 BYTECODE '12341ab8923c09efa932' )
> > <akarasulu> this can be an SC for example for checking for correct
> > social security numbers
> > <akarasulu> for example
> > <akarasulu> make sense?
> > <akarasulu> another representation may be ...
> > <elecharny> you already have these descriptions in RFC 4517
> > <akarasulu> ( 1.2.34.32.2.21 FQCN
> > org.apache.directory.server.schema.SocialSecuritySyntaxChecker )
> > <elecharny> ah, sorry, no
> > <elecharny> ok
> > <akarasulu> no no this is specific to ADS
> > <elecharny> yeah
> > <elecharny> my bad
> > <akarasulu> we're extending schema this way
> > <akarasulu> to allow dynamic changes
> > <akarasulu> no LDAP server on the market can do this
> > <elecharny> yeah yeah yeah
> > <akarasulu> i.e. to make SUN DS do this you have to write your own
> > plugins for it and restart it
> > <elecharny> ok
> > <elecharny> but then, what I said is that
> > <akarasulu> so we will have 100% extendible and dymamic schema
> > <akarasulu> shoot
> > <elecharny> as soon as you inject this ( 1.2.34.32.2.21 BYTECODE
> > '12341ab8923c09efa932' ) object
> > <akarasulu> ahha
> > <elecharny> where the bytecodes represent the compiled java class
> > <akarasulu> ahha
> > <elecharny> then you can extract out of it all the information regarding
> > the SC
> > <akarasulu> exactly
> > <elecharny> (if these bytcodes is a SC compiled class)
> > <elecharny> so no more need to inject the SC for that
> > <akarasulu> yeah we will do verification at the point this new
> > syntaxCheckers attribute value is added
> > <elecharny> because we can generate them on the fly too
> > <akarasulu> yeah exactly
> > <elecharny> let me explain
> > <elecharny> to see if we are on the same line
> > <akarasulu> ok listening
> > <elecharny> for instance, we want to inject the BooleanSyntaxChecker
> > object
> > <elecharny> (ok, it's already existing, but suppose we are creating it)
> > <akarasulu> well this is a built in
> > <akarasulu> ok ok
> > <akarasulu> np
> > <elecharny> we will compile the BooleanSyntaxChecker.java
> > <elecharny> producting some bytecode
> > <akarasulu> aha
> > <akarasulu> yep
> > <elecharny> and we will inject it into ADS
> > <akarasulu> yep
> > <elecharny> now, you have :
> > <elecharny> ( 1.3.6.1.4.1.1466.115.121.1.7 DESC 'Boolean' )
> > <elecharny> which is the definition of the Boolean SC in RFC 4517
> > <akarasulu> yeah this is the syntax not the SC ;)
> > <elecharny> (yes, but let me ficnish ;)
> > <akarasulu> this a value in the syntaxes attribute
> > <akarasulu> ok
> > <elecharny> what I mean is that now, you can read the compiled class
> > using introspection,
> > <elecharny> extract it's OID (which is stored,
> > <elecharny> and suppose we also added a desc member
> > <elecharny> we can extract the desc field,
> > <akarasulu> ahha
> > <elecharny> and inject into cn=schema this Boolean Syntax
> > <akarasulu> well I see what  you're thinking
> > <elecharny> we have just built from the bytecode
> > <akarasulu> take a step back for a second
> > <elecharny> so no need to have a Syntax Checker Parser, because we have
> > injected the binary object
> > <akarasulu> hmmm
> > <elecharny> from which we will be able to regenrate this SC syntax
> > <elecharny> ok, stepping back
> > <akarasulu> well you're mixing a few things together
> > <akarasulu> let me clarify this picture
> > <akarasulu> first a SC is not a Syntax
> > <akarasulu> looking at the protocol way to add schema elements
> > <elecharny> sorry, I was talking about syntax definitions
> > <akarasulu> you cannot add a new syntax to the syntaxes attribute unless
> > you have a SC for that syntax
> > <akarasulu> but technically you could just generate the syntax once you
> > have the SC
> > <elecharny> yes
> > <akarasulu> but this is mixing up two steps
> > <elecharny> this is what I had in mind
> > <akarasulu> hmmm let me think
> > <akarasulu> so you add the SC and that automatically generates the
> > Syntax for that SC
> > <elecharny> let's go back to basic
> > <akarasulu> is this what you mean?
> > <elecharny> you want to inject an entry
> > <akarasulu> wait don't go forward
> > <akarasulu> let's not mix up concepts
> > <elecharny> ok
> > <elecharny> sorry
> > <akarasulu> you're mechanism is a good idea that can basically reduce a
> > 2 step process into a 1 step process
> > <akarasulu> so instead of adding a SC then adding a S you just add an SC
> >
> > <akarasulu> this is what you mean?
> > <elecharny> what I wanted to say is that when you describe a new
> > attributeType, you have to describe the syntax it must respect
> > <elecharny> for instance, the attributeType :
> > <akarasulu> and use reflection to determine all the info for the Syntax
> > like it's DESC value etc ?
> > <elecharny> yeah
> > <akarasulu> yeah now you are mixing into AT stuff
> > <akarasulu> ok this will confuse us more
> > <akarasulu> let's finish SC verses S concept first
> > <akarasulu> IMHO this shit is very complex
> > <elecharny> I just wanted to say that you NEED to know the Syntax OID
> > before using it in an AttributeType
> > <akarasulu> yeah which means ...\
> > <akarasulu> you must add the syntax before you can add the AT
> > <elecharny> yeah
> > <akarasulu> but this is another subject
> > <elecharny> correct
> > <akarasulu> unreated to what we are currently discussing
> > <elecharny> true
> > <akarasulu> now SC vs. S
> > <akarasulu> I think this shit is complex and confusing
> > <elecharny> just wanted to be clear, but I'm just moving the mud :)
> > <elecharny> ok.
> > <akarasulu> let's not take shortcuts
> > <akarasulu> or overload meanings
> > <akarasulu> let's be mechanical about it
> > <elecharny> so in a standard approach, you first declare the S, and then
> >
> > load the SC
> > <akarasulu> hmmm no
> > <akarasulu> you first do SC
> > <akarasulu> then do S
> > <akarasulu> SC btw use OID of the S
> > <akarasulu> SC does not have it's own OID
> > <akarasulu> this is how the association takes place
> > <elecharny> so you must declare S before SC if Sc use S :)
> > <akarasulu> not really
> > <elecharny> SC is just the implementation
> > <akarasulu> remember S depends on SC in the object relationship sense
> > <akarasulu> yeah a Syntax object has a handle on the SC
> > <elecharny> so S comes first :)
> > <akarasulu> Syntax is a valid schema object with DESC and all the other
> > stuff
> > <akarasulu> well here's how we simplify it
> > <akarasulu> you add SC first
> > <akarasulu> then S
> > <akarasulu> you cannot add an S to the server before the SC
> > <akarasulu> here is why
> > <akarasulu> let me explain it has to do with the protocol
> > <elecharny> ahhh
> > <akarasulu> we cannot change the S syntax because this is mandated by
> > the protocol
> > <elecharny> I get it
> > <akarasulu> right?
> > <elecharny> I'm talking about specs, you are talking about loading those
> >
> > guys into ADS
> > <elecharny> in this way, I agree
> > <akarasulu> well I'm talking both
> > <elecharny> you first load SC,
> > <elecharny> and then  you declare S
> > <akarasulu> I am constrained by many things you're not thinking about at
> >
> > this point
> > <akarasulu> let me explain its important
> > <akarasulu> I want you to have this in mind fully
> > <akarasulu> basically
> > <akarasulu> ...
> > <akarasulu> (1) protocol demans a certain syntax for syntaxDescriptions
> > and we cannot change this
> > <akarasulu> (2) We cannot just add a new syntax using the protocol's
> > representation with a syntaxDescrition because it's not a functional
> > representation of a syntax.  Meaning we cannot do anything with (
> > 1.23.4.2 DESC 'blah blah blah' ).  The server cannot check values with
> > just this string.  It needs a syntaxChecker object associated with this
> > syntax to functionally enforce this new syntax.
> > <elecharny> yes, I know that.
> > <akarasulu> So SC must be added before an S can be added.
> > <akarasulu> If someone adds an S and there is no SC for the OID of that
> > S then the server must reject that add.
> > <akarasulu> because the server cannot enforce that syntax
> > <elecharny> yes
> > <akarasulu> this is why we have this break up of S and SC
> > <elecharny> ok.
> > <elecharny> but the strange thing about the protocol,
> > <elecharny> is taht it defines a syntax for S,
> > <akarasulu> because of both protocol limitations and the need to allow
> > for protocol based syntax updates that extend the server
> > <elecharny> but does bnot explain how to add a new S :)
> > <akarasulu> yeah the protocol sucks
> > <akarasulu> exactly
> > <elecharny> not a big deal, thoough
> > <akarasulu> protocol was not even remotely aware of how to dynamically
> > change the code of the server
> > <akarasulu> the guys who wrote it did not know about Java
> > <akarasulu> :D
> > <elecharny> let say it will be meaningfull for exposing the S when you
> > request the schema
> > <akarasulu> yeah exactly
> > <akarasulu> people will see that a syntax is defined for it
> > <elecharny> ok. At this point, we agree that if we can introspect SC, we
> > can generate S
> > <akarasulu> we can but IMHO this is bad
> > <akarasulu> I recommend absolutely no shortcuts or overloading of
> > actions or meanings for simplicity
> > <akarasulu> this stuff is too complex
> > <akarasulu> leave it so user must load SC then load S
> > <akarasulu> or load both at the same time
> > <akarasulu> do you agree ?
> > <elecharny> ok
> > <elecharny> not a big deal, though
> > <akarasulu> you may not in which case we can discuss it
> > <elecharny> we can ask the user to load SC then S
> > <elecharny> it's not really a PITA
> > <akarasulu> well hopefull we let LDAP studio do this
> > <elecharny> I just tried to see if we can avoid one of those two steps
> > <akarasulu> if it works with elcipse you can design and write your java
> > SC
> > <elecharny> yes
> > <akarasulu> and then LS will load it for u into live server
> > <elecharny> sure
> > <akarasulu> LS can make it seem like one step
> > <akarasulu> really for me I am looking to keep all functionality simple
> > <elecharny> ok
> > <akarasulu> not overloaded with meaning
> > <elecharny> I agree
> > <elecharny> the point is that then you will need thos efucking parsers
> > :)
> > <akarasulu> yeah it's ok
> > <akarasulu> the string is simple
> > <elecharny> (and I wanted to avoid writting them ;)
> > <elecharny> ok.
> > <elecharny> we just have to implement those parser in syntaxCheckers,
> > <akarasulu> well we need either an SC or S parser
> > <elecharny> where they are described in RFC 4517
> > <akarasulu> if we overload
> > <akarasulu> if not then we can write parsers for both S and SC
> > <elecharny> we don't have to write a SC parser
> > <akarasulu> I recommend doing it for both the syntax of these
> > descriptions are really simple
> > <akarasulu> yeah we do
> > <akarasulu> we need to write S and SC
> > <elecharny> the SC is the implementation
> > <akarasulu> no we need it because user can add a SC via the subentry
> > <elecharny> but what will be the SC syntax then ???
> > <akarasulu> ( 1.12.3.21.3 BYTECODE '23aae23a543bc343' ) for example
> > <elecharny> ahhh, it's OUR syntax
> > <akarasulu> yeah
> > <elecharny> ok ok ok
> > <akarasulu> there will be two forms
> > <elecharny> I thought you were talking about one existing RFC syntax
> > <akarasulu> one for existing classes in the server that come packaged
> > with ADS like so ...
> > <elecharny> ok, I agree.
> > <elecharny> man, I have to go.
> > <akarasulu>  ( 1.12.3.21.3 FQCN
> > org.apache.directory.server.XYZSyntaxChecker ) for example
> > <elecharny> can you copy/paste all this interesting convo and send it to
> >
> > the list  ?
> > <akarasulu> so one uses FQCN and other uses BYTECODE
> >
> >
> >
>
>
> --
> Ersin




-- 
Cordialement,
Emmanuel L├ęcharny

Mime
View raw message