From dev-return-39361-apmail-directory-dev-archive=directory.apache.org@directory.apache.org Wed Oct 19 19:27:56 2011 Return-Path: X-Original-To: apmail-directory-dev-archive@www.apache.org Delivered-To: apmail-directory-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C5FEF9AEF for ; Wed, 19 Oct 2011 19:27:56 +0000 (UTC) Received: (qmail 38641 invoked by uid 500); 19 Oct 2011 19:27:56 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 38610 invoked by uid 500); 19 Oct 2011 19:27:56 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 38603 invoked by uid 99); 19 Oct 2011 19:27:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Oct 2011 19:27:56 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of gokturk.gezer@gmail.com designates 209.85.210.46 as permitted sender) Received: from [209.85.210.46] (HELO mail-pz0-f46.google.com) (209.85.210.46) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Oct 2011 19:27:51 +0000 Received: by pzk2 with SMTP id 2so6065116pzk.5 for ; Wed, 19 Oct 2011 12:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=nLUsiYgtGscvGJsTZG7J17cZj6Cb4uMLwv0qJgxx1uU=; b=GJNasqML+jxfowztPay3+09i9NvF2sylSsWeoDsrvRnvnsIw2+Cio4qXt+Dsvkx1fX VLk5ES2nVxM4oLSXWNW9jfa6Xzg0rQBiq2K+Hh264Rytr13RkSHg1ePUYMocV+tUia63 uzY49joklG1N91L+PY1ym6RItu3qKwWKn2zkA= MIME-Version: 1.0 Received: by 10.68.36.103 with SMTP id p7mr14264104pbj.74.1319052449297; Wed, 19 Oct 2011 12:27:29 -0700 (PDT) Received: by 10.68.43.69 with HTTP; Wed, 19 Oct 2011 12:27:29 -0700 (PDT) In-Reply-To: <-5308071789035679513@unknownmsgid> References: <-5308071789035679513@unknownmsgid> Date: Wed, 19 Oct 2011 22:27:29 +0300 Message-ID: Subject: Re: [API] Experimenting OSGI fragments to solve the extensibility issue From: =?UTF-8?B?R8O2a3TDvHJrIEdlemVy?= To: Apache Directory Developers List Content-Type: multipart/alternative; boundary=bcaec51dd5cbae344604afabd412 --bcaec51dd5cbae344604afabd412 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, Oct 19, 2011 at 10:13 PM, Pierre-Arnaud Marcelot wrote: > Hi G=C3=B6kt=C3=BCrk, > > Le 19 oct. 2011 =C3=A0 19:05, "G=C3=B6kt=C3=BCrk Gezer" a > =C3=A9crit : > > Hi Pierre, > > On Wed, Oct 19, 2011 at 6:36 PM, Pierre-Arnaud Marcelot = wrote: > >> Hi Dev, >> >> Following our efforts on the OSGI side, I took some time to think about >> (and experiment) how we could solve our extensibility issue on the LDAP = API, >> where we want our users to be able to provide their own custom >> implementation of various schema objects like comparators, normalizers, = etc. >> >> After a quick discussion with Emmanuel and Guillaume Nodet, it turned ou= t >> using OSGI fragments bundles could probably be our best solution. >> > Hmm, may not be the best, see below. > >> >> Here's a definition of what an OSGI fragment bundle is (more information >> available at [1]): >> "An OSGi fragment is a Java archive file with specific manifest headers >> that enable it to attach to a specified host bundle or specified host >> bundles in order to function. >> Fragments are treated as part of the host bundles. Relevant definitions = of >> the fragment are merged with the host bundles definitions before the hos= t is >> resolved, as long as the information does not conflict. >> Fragment dependencies are resolved if possible. If the fragment >> dependencies can not be resolved, the fragment does not attach to the ho= st >> bundle. >> A fragment can not have its own class loader or bundle activator. It can >> not override the information present in the host bundles. Fragments exte= nd >> bundles with resources, classes, and permitted headers enabling you to >> customize your bundles." >> > Yes, fragments are merged with their host and they left up in the framewo= rk > as just resolved, no active state at all. Everything it imports/exports > become host's imports/exports. > >> >> The great thing about fragments is that they *share* the same class load= er >> as their host bundle. Which was pretty much the kind of issue we were ha= ving >> with classloaders being differents from one bundle to the other. >> > >> > >> The other great thing I see with this approach, it that it would also wo= rk >> great outside of an OSGI container application (which is a strong >> requirement for the LDAP API). A fragment bundle is nothing else than a >> regular bundle with a specific OSGI directive (Fragment-Host) added to i= ts >> MANIFEST.MF file, and a bundle behaves exactly like a plain jar file whe= n >> it's not included in an OSGI container. Thus, it would allow us to suppo= rt >> third parties extensions. >> > It would only allow 3th parties to support their own extensions. Why?. > Because being able to classload the 3th party classes is only half of the > story. We must also know that they've arrived and we must know what has b= een > arrived. Otherwise we have to hard code their name, ant it is not case in > 3th party scenario. Once the fragment is merged with the host, we can cla= ss > load them inside host and some other bundle having access to that fragmen= ts > specific package? But how do we enforce that package names and class name= s > across 3th parties? > > > Actually, that's exactly how it works at the moment. The fully qualified > name of the class is available and read from the schema ldif files. That'= s > how we get the other half of the story. > If someone wants to extend the schema with a new comparator, for example, > it needs to provide the class and edit the schema to declare the new > comparator. > Hmm. yes i see now. Actually this is a problem of Studio more than DS. Because it is used to create schemas. So when some 3th party provide custom elements, its Studio's responsibility to find and list it under corresponding GUI element. After that, DS can load it using that class name either with classloading(with fragments) or OSGI. > > I have done a small experiment in my Eclipse workspace with two bundles >> (one being the host and the other being the fragment ) and I was able to >> classload, without any classloader issue, a class defined in the fragmen= t >> bundle from another class inside the host bundle. >> > Did you do it while knowing the class name on the fragment bundle, or not > knowing the class name at all? > > > Yep, indeed. > > I'd like to go a step further and experiment this on the API itself. >> >> Ideally, I'd like to use the 'shared-ldap-model' module >> ('org.apache.directory.shared.ldap.model' bundle) as a host for all our >> schema objects implementations and move these implementations into a >> specific fragment bundle containing them all. >> Users would have to do the same to include their extensions. A simple >> fragment bundle with the 'org.apache.directory.shared.ldap.model' bundle= as >> host and it works. >> > Yes it works for us, but for some body writing his own classes, we must > dynamically know they've arrived. > >> >> > >> One other thing that should be done, is to let the 'shared-ldap-model' >> module do the instanciation of the schema elements. At the moment, two >> classes from the 'shared-ldap-schema-data' module are responsible for th= is, >> 'org.apache.directory.shared.ldap.schemaloader.SchemaEntityFactory' and >> 'org.apache.directory.shared.ldap.schemaloader.AttributeClassLoader'. We >> would need to move these classes to the 'shared-ldap-model' module, for = them >> to have access to the right class loader. >> >> All in all, I think that's the only things we need to do to get the >> extensibility we wanted inside and outside an OSGI container. >> > It may bring us to somewhere but not even close to real goals IMO. We can > use them to provide in-house extensions, but when it comes to 3th parties= , > it won't be so easy to use that fragments comnig from them inside Shared = or > ApacheDS. > > > Sure, this is some very basic extensibility but it matches perfectly our > needs for the API. On the ApacheDS side, it's another story and we might > probably need something better and more complex. > I was already scared to play with shared anyway, now my fear tripled :) But i have some good news. I made my use of IPojo and OSGI to not broke existin= g shared clients. But i've got one more issue: the OIDs that's got assigned b= y constructors, are they static? or is it possible to lets say BooleanComparator to be instantiated with two different oids. As I see its not the case and you're maintaining standart OID assigments by hand. What i= s the case here? > > Regards, > Pierre-Armaud > > What do you think of this plan? >> >> Regards, >> Pierre-Arnaud >> >> [1] - >> http://publib.boulder.ibm.com/infocenter/radhelp/v8/index.jsp?topic=3D/c= om.ibm.osgi.common.doc/topics/cbundlefragment.html >> > > Regards, > Gokturk > > Regards, Gokturk --bcaec51dd5cbae344604afabd412 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Oct 19, 2011 at 10:13 PM, Pierre= -Arnaud Marcelot <= pajbam@gmail.com> wrote:
Hi=C2=A0G=C3=B6kt=C3=BCrk,

Le 19 oct. 2011 =C3=A0 19:05, "G=C3=B6kt=C3=BCrk Gezer" &l= t;gokturk.geze= r@gmail.com> a =C3=A9crit=C2=A0:

Hi Pi= erre,

On Wed, Oct 19, 2011 at 6:36 PM, Pi= erre-Arnaud Marcelot <pa@marcelot.net> wrote:
Hi Dev,

Following ou= r efforts on the OSGI side, I took some time to think about (and experiment= ) how we could solve our extensibility issue on the LDAP API, where we want= our users to be able to provide their own custom implementation of various= schema objects like comparators, normalizers, etc.

After a quick discussion with Emmanuel and Guillaume No= det, it turned out using OSGI fragments bundles could probably be our best = solution.
Hmm, may not be the best, see below.= =C2=A0
Here's a definition of what an OSGI fragment bundle=C2=A0is= =C2=A0(more information available at [1]):
"An OSGi fragment is a Java archive file with specific manifest h= eaders that enable it to attach to a specified host bundle or specified hos= t bundles in order to function.
Fragments are treated as part of = the host bundles. Relevant definitions of the fragment are merged with the = host bundles definitions before the host is resolved, as long as the inform= ation does not conflict.
Fragment dependencies are resolved if possible. If the fragment depend= encies can not be resolved, the fragment does not attach to the host bundle= .
A fragment can not have its own class loader or bundle activato= r. It can not override the information present in the host bundles. Fragmen= ts extend bundles with resources, classes, and permitted headers enabling y= ou to customize your bundles."
Yes, fragments are merged with their host and they = left up in the framework as just resolved, no active state at all. Everythi= ng it imports/exports become host's imports/exports.

The great thing abo= ut fragments is that they *share* the same class loader as their host bundl= e. Which was pretty much the kind of issue we were having with classloaders= being differents from one bundle to the other.=C2=A0
=C2=A0

The other great thi= ng I see with this approach, it that it would also work great outside of an= OSGI container application (which is a strong requirement for the LDAP API= ). A fragment bundle is nothing else than a regular bundle with a specific = OSGI directive (Fragment-Host) added to its MANIFEST.MF file, and a bundle = behaves exactly like a plain jar file when it's not included in an OSGI= container. Thus, it would allow us to support third parties extensions.
It would only allow 3th parties to support their ow= n extensions. Why?. Because being able to classload the 3th party classes i= s only half of the story. We must also know that they've arrived and we= must know what has been arrived. Otherwise we have to hard code their name= , ant it is not case in 3th party scenario. Once the fragment is merged wit= h the host, we can class load them inside host and some other bundle having= access to that fragments specific package? But how do we enforce that pack= age names and class names across 3th parties?

Actually, that's exa= ctly how it works at the moment. The fully qualified name of the class is a= vailable and read from the schema ldif files. That's how we get the oth= er half of the story.=C2=A0
If someone wants to extend the schema with a new comparator, for examp= le, it needs to provide the class and edit the schema to declare the new co= mparator.=C2=A0
Hmm. yes i see now. Actually t= his is a problem of Studio more than DS. Because it is used to create schem= as. So when some 3th party provide custom elements, its Studio's respon= sibility to find and list it under corresponding GUI element. After that, D= S can load it using that class name either with classloading(with fragments= ) or OSGI.
=
I h= ave done a small experiment in my Eclipse workspace with two bundles (one b= eing the host and the other being the fragment ) and I was able to classloa= d, without any classloader issue, a class defined in the fragment bundle fr= om another class inside the host bundle.
Did you do it while knowing the class name on the f= ragment bundle, or not knowing the class name at all?=C2=A0

Yep, indeed.=C2=A0

I'd like to go a step further = and experiment this on the API itself.

Ideally, I&= #39;d like to use the 'shared-ldap-model' module ('org.apache.d= irectory.shared.ldap.model' bundle) as a host for all our schema object= s implementations and move these implementations into a specific fragment b= undle containing them all.
Users would have to do the same to include their extensions. A simple = fragment bundle with the=C2=A0'org.apache.directory.shared.ldap.model&#= 39; bundle as host and it works.
Yes it works = for us, but for some body writing his own classes, we must dynamically know= they've arrived.=C2=A0
=C2= =A0

One other thing tha= t should be done, is to let=C2=A0the 'shared-ldap-model' module do = the instanciation of the schema elements. At the moment, two classes from t= he 'shared-ldap-schema-data' module are responsible for this, '= org.apache.directory.shared.ldap.schemaloader.SchemaEntityFactory' and = 'org.apache.directory.shared.ldap.schemaloader.AttributeClassLoader'= ;. We would need to move these classes to the=C2=A0'shared-ldap-model&#= 39; module, for them to have access to the right class loader.

All in all, I think that's the only things we need = to do to get the extensibility we wanted inside and outside an OSGI contain= er.
It may bring us to somewhere but not even = close to real goals IMO. We can use them to provide in-house extensions, bu= t when it comes to 3th parties, it won't be so easy to use that fragmen= ts comnig from them inside Shared or ApacheDS.=C2=A0

Sure, this is some very = basic extensibility but it matches perfectly our needs for the API. On the = ApacheDS side, it's another story and we might probably need something = better and more complex.=C2=A0
I was already scared to play with shared anyway, no= w my fear tripled :) But i have some good news. I made my use of IPojo and = OSGI to not broke existing shared clients. But i've got one more issue:= the OIDs that's got assigned by constructors, are they static? or is i= t possible to lets say BooleanComparator to be instantiated with two differ= ent oids. As I see its not the case and you're maintaining standart OID= assigments by hand. What is the case here?

Regards,
Pierre-Armaud


Regards,
Gokturk
Regards,
Gokturk
--bcaec51dd5cbae344604afabd412--