Return-Path: Delivered-To: apmail-directory-dev-archive@www.apache.org Received: (qmail 66675 invoked from network); 31 Jan 2011 14:00:22 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 31 Jan 2011 14:00:22 -0000 Received: (qmail 54852 invoked by uid 500); 31 Jan 2011 14:00:22 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 54655 invoked by uid 500); 31 Jan 2011 14:00:19 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 54645 invoked by uid 99); 31 Jan 2011 14:00:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Jan 2011 14:00:18 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of akarasulu@gmail.com designates 74.125.82.44 as permitted sender) Received: from [74.125.82.44] (HELO mail-ww0-f44.google.com) (74.125.82.44) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Jan 2011 14:00:13 +0000 Received: by wwa36 with SMTP id 36so5810014wwa.1 for ; Mon, 31 Jan 2011 05:59:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=CS4Wpuhv4z99+MFIZYFx4k+1+o0u1tNzsjBvwJ0c7UE=; b=jHqe29cWKA5tC5DIH366gjizjmVQCBxZjCAa19PPNtksIznOZ/wQwt+D/At6ox457X z5w7SKJhGmCEInAV/xtxr1sFR+Re9mlXywlieQe/m4BwKUFayThg2m0nSVu8xd5Y+avW cNP8uNwvymjWlt1oB02wjOUZCyM65HR1V7QYk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=RiN8reWQLvq0k4b4MQfwoR9zWjjodpm7evORYv17wDoQRJZTsLewGHA3kRLXBIR5fM 5pbDnwenqUeB6QQ4tmzcwMP5eLS8ZJx9m3ztsfFvWczW9s46XjkoeHuNdxc+yf7/HKkK 2VYyu8zQnFRU2CsZku5fnF12Rg6inGXV8Vl3A= MIME-Version: 1.0 Received: by 10.216.162.70 with SMTP id x48mr10910662wek.4.1296482391453; Mon, 31 Jan 2011 05:59:51 -0800 (PST) Sender: akarasulu@gmail.com Received: by 10.216.73.78 with HTTP; Mon, 31 Jan 2011 05:59:51 -0800 (PST) In-Reply-To: <4D46A767.1040805@gmail.com> References: <4D46981E.5060907@gmail.com> <4D46A767.1040805@gmail.com> Date: Mon, 31 Jan 2011 15:59:51 +0200 X-Google-Sender-Auth: Ss6kEa087uU78fcYnMRYEFxZY8k Message-ID: Subject: Re: Control interface From: Alex Karasulu To: Apache Directory Developers List , elecharny@apache.org Content-Type: multipart/alternative; boundary=001485f44f3266819b049b24d419 --001485f44f3266819b049b24d419 Content-Type: text/plain; charset=ISO-8859-1 Before going into each of your individual comments I simply want to reflect on my thoughts regarding the matter of Control's exposing their opaque values. First let me list some of the facts we're aware of: (1) LDAP controls contain an optional value payload encased in a surrounding envelop that holds the type (OID) and criticality: Control ::= SEQUENCE { controlType LDAPOID, criticality BOOLEAN DEFAULT FALSE, controlValue OCTET STRING OPTIONAL } (2) This optional value can be anything. It does not have to be ASN.1 encoded content. It can be JSON text, XML, or anything else. It's application specific. However in practice, of the known published controls the value is AFAIK 100% ASN.1. (3) Regardless of what is stored in the value, no application can just use it as is without some kind of component to decode, parse, or interpret this binary data. It's useless to API users otherwise. It's opaque according to LDAP, but there has to be some meaning to make use of this content, and that always requires some code to convert that binary data into usable information. A useful representation to an API user is a parameterized representation making the opaque structure transparent. What exact is that you may ask? Well it's a well defined data type with members and fields that can be accessed to drive handling logic. So regardless of whether this binary payload (the value), is json, xml, or ASN.1 BER encoded data, optimally we should present, or allow them to present this information to users as a transparent data structure, that they can use directly to drive code logic. There's not much you can directly do with a byte[]. (4) In JNDI land, the value, is left as the value, a byte[] because these guys were so busy with all sorts of naming systems, LDAP only got a little attention. The general API filled with compromises turned out to be problematic for the LDAP namespace. There is a way to make controls pluggable but it's a nightmare. In the end, the pass through strategy made sense for them. It's just pass through to get a handle on the content, deferring how to handle it to the application code. --- o --- I'll try to make some sense out of my disorderly thoughts now. (1) The application coder writing the logic to use the control, may not be the same entity writing/designing the control. The application needs a transparent parameterized view of the binary opaque data to do anything sensible with it. Using the data just in pass through situations may occur but it's the exception and not the rule. (2) Regardless of the binary value encoding (i.e. json, xml, BER), it can be marshaled into a distinct data representation that makes sense for application coders to use. So if a specific Control subtype in addition to the type (OID) and criticality properties, exposed all properties needed to act on that control's content, then these properties can be populated. This way it's actionable data that can directly be used instead of some useless binary payload. (3) It's up to control designers/implementers, to declare what the structure of their control is. They do this through ASN.1 for BER encoded content for example. For our codec they should so so by providing a POJI and simple POJO implementation of their control extending our Control interface which only has the properties: oid, and criticality. (4) Expose the POJO and POJI to control users. Yet hide the details of encoding and decoding. This really does not matter to the control user who just wants to implement some application logic. It matters not to him if the data is encoded as JSON, XML, or ASN.1 BER. Just just wants to make the application logic function based on the information inside. Expose this yet hide the implementation, (encoding) details. (5) Make a pluggable framework which supports #4. Let the control implementor declare (POJI/POJO), and implement (components for encoding and decoding content) the control's codec extension. The end result to users is the same regardless of the implementation details. (6) API's like JNDI left the pass thru door open but did it by exposing the implementation and potentially allowing the user to shoot themselves in the foot. Exposing the value is what I am referring to. Really that's fine if that's all you're going to do but decide on one path. Either expose it and don't bother representing the opaque data as actionable transparent typed information. Or don't expose it at all and just let application code deal with the logical typed information. So these are my points. Going inline now to answer questions or comment on remarks: On Mon, Jan 31, 2011 at 2:13 PM, Emmanuel Lecharny wrote: > On 1/31/11 1:05 PM, Stefan Seelmann wrote: > >> On Mon, Jan 31, 2011 at 12:08 PM, Emmanuel Lecharny >> wrote: >> >>> Hi, >>> >>> in the new Control interface we are now using (not the JNDI one), we >>> don't >>> expose the getValue() setValue() hasValue() methods. In many respects, >>> those >>> methods are useless, and we don't use them in the server. >>> >>> +1 To us we don't care or should not care about this value. We're application layer once we try to make sense of the information contained in the value. We don't care about it as opaque data in studio and in the server. > However, from a user perspective, someone writing some code on top of the >>> LDAP API, connected to an unknown server, and getting back some unknown >>> control, will have no way to get the inner values, unless casting the >>> control using the ControlDecorator interface. >>> >>> This is wrong. I think we must expose the three methods to the world. >>> >> If the user receives an unknown control, what can he do with the value >> then? He must be able to decode the ASN.1 BER encoded value anyway, >> right? Then he can also write a new Control class. Am I missing >> something? >> > So yeah the byte[] data in the end is useless to everyone. In the end it must be interpreted to take some action. You know when you get the OID if you can take action anyway. If you cannot then what's the point to decoding the value content. > He can write a new Control class, but he can also get the opaque value and > send it to another server. > > I off the top of my head cannot think of very many use cases for this limited situation but we should presume everything is possible. But in this case, the opaque data is not lost if it's coming in from somewhere, yet it should still not be exposed according to the reasoning I documented above. So in this scenario, yes some opaque data comes in and you're not going to do anything but pass it thru to something else. Ok so you get a Control that does not expose the value. Under the hood it's a ControlDecorator that holds the value. So the value is available without exposing application user access to it. Then when the control needs to be sent off to somewhere else, this data can be leveraged in the codec, at the layer that should be aware of it, to prep it for the hand off. Is this making sense? I may be long winded. Still trying to regain consciousness this morning. I mean, there are probably use cases where we want to get the value, not > decoding it, and use it to do something else. > Yes, but then why bother exposing the value. It's useless and can cause trouble is what my line of thinking has lead me to. > > Should we force the users to write their own Controls in such a case? Isn't > it better to provide a generic control with accessors for the value? Ins't > it a bit over killing to ask the LdapAPI users to implement their own > controls in order for them to manipulate this value? > > (1) In the end if the user is going to use the value and take steps in their application logic based on that content, then they'd have to decode the value themselves whatever it is anyway. (2) If it's opaque, gonna stay opaque, users need not know what's inside, and it's going to be shuffled off (passed thru) to some other code, client, server whatever without specific consideration for the content, then there should be no reason the user code needs access to it. Let the codec encode decode it, and let only the codec access the binary opaque data. The alternative is to implement all known controls into the API... > > I think if we have a sound plugin mechanism that's easy to use controls can be implemented very rapidly. Otherwise even the pass thru mechanism has little to no value. Practically, after removing the value property, we had serious hardships shifting our code which was so dependent on accessing this opaque content as is. I think this is due to the fact that there was no other way to effectively deal with this at that time. Regards, -- Alex Karasulu My Blog :: http://www.jroller.com/akarasulu/ Apache Directory Server :: http://directory.apache.org Apache MINA :: http://mina.apache.org To set up a meeting with me: http://tungle.me/AlexKarasulu --001485f44f3266819b049b24d419 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Before going into each of your individual comments I simply want to reflect= on my thoughts regarding the matter of Control's exposing their opaque= values.=A0

First let me list some of the facts we'r= e aware of:

(1) LDAP controls contain an optional value payload enc= ased in a surrounding envelop that holds the type (OID) and criticality:

        Control ::=3D SEQUENCE {
                controlType             LDAPOID,
                criticality             BOOLEAN DEFAULT FALSE,
                controlValue            OCTET STRING OPTIONAL }

(2) This optional value can be anything. It does= not have to be ASN.1 encoded content. It can be JSON text, XML, or anythin= g else. It's application specific. However in practice, of the known pu= blished controls the value is AFAIK 100% ASN.1.=A0

(3) Regardless of what is stored in the value, no appli= cation can just use it as is without some kind of component to decode, pars= e, or interpret this binary data. It's useless to API users otherwise. = It's opaque according to LDAP, but there has to be some meaning to make= use of this content, and that always requires some code to convert that bi= nary data into usable information. A useful representation to an API user i= s a parameterized representation making the opaque structure transparent.= =A0

What exact is that you may ask? Well it's a well de= fined data type with members and fields that can be accessed to drive handl= ing logic.=A0

So regardless of whether this binary= payload (the value), is json, xml, or ASN.1 BER encoded data, optimally we= should present, or allow them to present this information to users as a tr= ansparent data structure, that they can use directly to drive code logic. T= here's not much you can directly do with a byte[].

(4) In JNDI land, the value, is left as the value, a by= te[] because these guys were so busy with all sorts of naming systems, LDAP= only got a little attention. The general API filled with compromises turne= d out to be problematic for the LDAP namespace. There is a way to make cont= rols pluggable but it's a nightmare. In the end, the pass through strat= egy made sense for them.=A0It's just pass through to get a handle on th= e content, deferring how to handle it to the application code.

--- o = ---

I&= #39;ll try to make some sense out of my disorderly thoughts now.=A0

(1) The application coder writing the = logic to use the control, may not be the same entity writing/designing the = control. The application needs a transparent parameterized view of the bina= ry opaque data to do anything sensible with it. Using the data just in pass= through situations may occur but it's the exception and not the rule.= =A0

(2) Regardl= ess of the binary value encoding (i.e. json, xml, BER), it can be marshaled= into a distinct data representation that makes sense for application coder= s to use. =A0So if a specific Control subtype in addition to the type (OID)= and criticality properties, exposed all properties needed to act on that c= ontrol's content, then these properties can be populated. This way it&#= 39;s actionable data that can directly be used instead of some useless bina= ry payload.

(3) It'= s up to control designers/implementers, to declare what the structure of th= eir control is. They do this through ASN.1 for BER encoded content for exam= ple. For our codec they should so so by providing a POJI and simple POJO im= plementation of their control extending our Control interface which only ha= s the properties: oid, and criticality.

(4) Expose = the POJO and POJI to control users. Yet hide the details of encoding and de= coding. This really does not matter to the control user who just wants to i= mplement some application logic. It matters not to him if the data is encod= ed as JSON, XML, or ASN.1 BER. Just just wants to make the application logi= c function based on the information inside. Expose this yet hide the implem= entation, (encoding) details.

(5) Make a = pluggable framework which supports #4. Let the control implementor declare = (POJI/POJO), and implement (components for encoding and decoding content) t= he control's codec extension. The end result to users is the same regar= dless of the implementation details.

(6) API'= ;s like JNDI left the pass thru door open but did it by exposing the implem= entation and potentially allowing the user to shoot themselves in the foot.= Exposing the value is what I am referring to. Really that's fine if th= at's all you're going to do but decide on one path. Either expose i= t and don't bother representing the opaque data as actionable transpare= nt typed information. Or don't expose it at all and just let applicatio= n code deal with the logical typed information.

So these ar= e my points. Going inline now to answer questions or comment on remarks:

On Mon, = Jan 31, 2011 at 2:13 PM, Emmanuel Lecharny <elecharny@gmail.com> wrote:
On 1/31/1= 1 1:05 PM, Stefan Seelmann wrote:
On Mon, Jan 31, 2011 at 12:08 PM, Emmanuel Lecharny<elecharny@gmail.com> =A0wrote:<= br>
Hi,

in the new Control interface we are now using (not the JNDI one), we don= 9;t
expose the getValue() setValue() hasValue() methods. In many respects, thos= e
methods are useless, and we don't use them in the server.


= +1

To us we don't care or should not care abou= t this value. We're application layer once we try to make sense of the = information contained in the value. We don't care about it as opaque da= ta in studio and in the server.=A0
=A0
However, from a user perspective, someone writing some code on top of the LDAP API, connected to an unknown server, and getting back some unknown
control, will have no way to get the inner values, unless casting the
control using the ControlDecorator interface.

This is wrong. I think we must expose the three methods to the world.
If the user receives an unknown control, what can he do with the value
then? He must be able to decode the ASN.1 BER encoded value anyway,
right? Then he can also write a new Control class. Am I missing
something?

So = yeah the byte[] data in the end is useless to everyone. In the end it must = be interpreted to take some action. You know when you get the OID if you ca= n take action anyway. If you cannot then what's the point to decoding t= he value content.
=A0
He can write a new Control class, but he can also get the opaque value and = send it to another server.


I off the top of my head cannot think = of very many use cases for this limited situation but we should presume eve= rything is possible. But in this case, the opaque data is not lost if it= 9;s coming in from somewhere, yet it should still not be exposed according = to the reasoning I documented above.=A0

So in this scenario, yes some opaque data comes in and = you're not going to do anything but pass it thru to something else. Ok = so you get a Control that does not expose the value. Under the hood it'= s a ControlDecorator that holds the value. So the value is available withou= t exposing application user access to it. Then when the control needs to be= sent off to somewhere else, this data can be leveraged in the codec, at th= e layer that should be aware of it, to prep it for the hand off.

Is this making sense? I may be long winded. Still tryin= g to regain consciousness this morning.

I mean, there are probably use cases where we want to get the value, not de= coding it, and use it to do something else.

=
Yes, but then why bother exposing the value. It's useless and can = cause trouble is what my line of thinking has lead me to.
=A0

Should we force the users to write their own Controls in such a case? Isn&#= 39;t it better to provide a generic control with accessors for the value? I= ns't it a bit over killing to ask the LdapAPI users to implement their = own controls in order for them to manipulate this value?


(1) In the end if the user is going to= use the value and take steps in their application logic based on that cont= ent, then they'd have to decode the value themselves whatever it is any= way.
=A0
(2) If it's opaque, gonna stay opaque, users need no= t know what's inside, and it's going to be shuffled off (passed thr= u) to some other code, client, server whatever without specific considerati= on for the content, then there should be no reason the user code needs acce= ss to it. Let the codec encode decode it, and let only the codec access the= binary opaque data.=A0

The alternative is to implement all known controls into the API...


I= think if we have a sound plugin mechanism that's easy to use controls = can be implemented very rapidly. Otherwise even the pass thru mechanism has= little to no value.

Practically, after removing the value property, we had= serious hardships shifting our code which was so dependent on accessing th= is opaque content as is. I think this is due to the fact that there was no = other way to effectively deal with this at that time.=A0

--001485f44f3266819b049b24d419--