Return-Path: X-Original-To: apmail-directory-dev-archive@www.apache.org Delivered-To: apmail-directory-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 880D672F5 for ; Wed, 23 Nov 2011 16:37:05 +0000 (UTC) Received: (qmail 28004 invoked by uid 500); 23 Nov 2011 16:37:05 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 27964 invoked by uid 500); 23 Nov 2011 16:37:05 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 27957 invoked by uid 99); 23 Nov 2011 16:37:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Nov 2011 16:37:05 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of gokturk.gezer@gmail.com designates 209.85.213.50 as permitted sender) Received: from [209.85.213.50] (HELO mail-yw0-f50.google.com) (209.85.213.50) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Nov 2011 16:36:58 +0000 Received: by ywb26 with SMTP id 26so568324ywb.37 for ; Wed, 23 Nov 2011 08:36:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=O9S784sO/IVhPkWwMJRADJa3/BZrwbFWpBw/VrmQ6bs=; b=ECuMQj6wa7zWg6Rtb2aLiyMRoOgpWkNpFp3Pmc4QZnrFnFVGTwyXYx3uj/or0FQQAu PvwFILXTbLmJuZL1nGcG8eF0EVGejCC9xlaAga/l+7catz2pU+6LWfrW5pNf4lRH0RWb SkWDJSf4d8KCu/1AyHteDbV80pO/eFP8wwAKM= MIME-Version: 1.0 Received: by 10.68.20.234 with SMTP id q10mr9561906pbe.27.1322066196978; Wed, 23 Nov 2011 08:36:36 -0800 (PST) Received: by 10.68.22.229 with HTTP; Wed, 23 Nov 2011 08:36:36 -0800 (PST) In-Reply-To: <84E54BB5-1A87-42D1-8777-68FD237BEB1C@marcelot.net> References: <84E54BB5-1A87-42D1-8777-68FD237BEB1C@marcelot.net> Date: Wed, 23 Nov 2011 18:36:36 +0200 Message-ID: Subject: Re: [API][ApacheDS][Studio] Status on the OSGI branch From: =?UTF-8?B?R8O2a3TDvHJrIEdlemVy?= To: Apache Directory Developers List Content-Type: multipart/alternative; boundary=bcaec5215ddd0a5da004b26986fd X-Virus-Checked: Checked by ClamAV on apache.org --bcaec5215ddd0a5da004b26986fd Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, Nov 23, 2011 at 5:31 PM, Pierre-Arnaud Marcelot wr= ote: > Hi guys, > > I wanted to give you a little bit of feedback and ask for status from the > other team members (especially G=C3=B6kt=C3=BCk) working on the OSGI bran= ch. > > > On the API side, G=C3=B6kt=C3=BCk introduced extensibility for the compar= ators and > syntax checkers using Apache Felix iPojo. > From what I've seen it works great both as OSGI bundles and plain old jar= s. > That's cool. > > On the Studio side, I'm still struggling with these modifications. > Despite all my efforts and hacks (especially with the > 'Bundle-ActivationPolicy' directive of the MANIFEST.MF file), I'm still > unable to load extensible elements of the API within Studio. > > Here's a quick recap of the problems I encountered. > > At first, I noticed that in Equinox (Eclipse's OSGI container), the iPojo > bundle and Shared's iPojo utilities bundle were not started and their > activators not called when I was using the API > > I rebranded the iPojo bundle in Studio's projects to include the > 'Bundle-ActivationPolicy' directive in its MANIFEST.MF file, and also > included it in the 'ipojo-manager' of ApacheDS. > > This is already a bad hack, but it is still not enough=E2=80=A6 > > Now, both bundles are started at the first time one of their classes is > accessed by another bundle, and their activators are getting called > correctly. > > However I'm still getting an exception, a NullPointerException. > This NPE occurs when we ask the bundle context for the service references > related to iPojo. > Here's a file containing the full stack trace of the exception: [1] > > At this point, there should be a service reference for iPojo ready to use= , > but we get nothing. > > This is how far I've managed to go=E2=80=A6 It is not enough, but I've tr= ied every > fix idea I had and I empty now=E2=80=A6 > If someone has a clever idea=E2=80=A6 ;) > I just have ideas on why this is failing. The first thing to consider is, something in eclipse equinox is preventing IPojo from actually initializing. I mean it seems started but, actually it isn't. IPojo does not have custom equinox console command, so that makes finding out what happened harder. If we assume IPojo is initialized correctly, then the problem is about the thread model of the equinox which leads Ipojo annotated api elements(Comprators, etc...) to not registered with IPojo before the code looks for them through IPojo. Starting them right after the IPojo in equinox may solve the problem. Actually there is no nice-looking solution to this problem, because we're deploying osgi partially into the code, so we must manage these problems somehow. I couldn't have a chance to look at what is wrong with the studio. IPojo runs on equinox for sure. I'll try to set my environment for debugging that problem. > > On the ApacheDS side, I am able to run a small part of ApacheDS using the > 'apache-felix' module and its Eclipse launch configuration. > > At the end of the execution I hit this exception: > org.apache.directory.server.config.ConfigurationException: Cannot > initialize the > org.apache.directory.server.core.normalization.NormalizationInterceptor, > error : java.lang.ClassNotFoundException: > org.apache.directory.server.core.normalization.NormalizationInterceptor n= ot > found by org.apache.directory.server.service.builder [44] > Here's a file containing the full stack trace of the exception: [2] > > I think it is, at the moment, the intended behavior since a few things > still remain to be set in ApacheDS to allow it to run successfully within > an OSGI container. > > G=C3=B6kt=C3=BCk, can you confirm this is the intended behavior please? > Yeah, that's the same stacktrace with the one i get . Interceptors are not even visible to service-osgi bundle, so NoClassDefFound is expected exception when we try to classload them which is what we do right now. More importantly, can you share with us the status of the OSGI-fication of > ApacheDS? What is the purpose of the 'component-hub' you have been workin= g > on lately? > component-hub is the main extendibility layer i'm working on. This will provide us as developers a facility to provide extendibility to any aspect of the ApacheDS. It's almost done with the implementation. It needs testing after that of course. It will be a standalone engine in ApacheDS, means you won't have to use OSGI directly to provide extendibility. ComponentHub provides facilities for, listening some type of components, creating instance of them, configuring the instances through associated entries in DIT, saving and loading those created/configured instances through java property files. So when we want to provide extendibility for interceptors lets say, we will write 3 class(schema generator, instance generator and listener). Schema generator is for generating custom schema elements to represent the component's instances on DIT. Instance generator in the other hand must be compatible with its schema generator, and its purpose is generating instances with the default configuration which we'll extract from the implementing class. When we register schema generator and instance generator with the component-hub, our listener will start getting information of all the interceptors in the system through callback methods with ADSComponent argument. Through this ADSComponent reference we'll be able to create manage,create.configure the component instances, easilly. Each created instance will also be represented by ADSComponentInstance reference which will provide facilities for hooking its configuration with the some entry on DIT and so..I already wrote general purpose schema-instance generators, but individual component types may need some extra, hardcoded things in its schema and DIT entries, so that's why i make generators extendible. I'll document the infrastructure after i finish and test it completely. All i can say is after it finishes, making something in ApacheDS extendible in all aspects will be really easy procedure. > > > I hope we can move forward and find fixes to all the remaining issues > quickly on OSGI. > Even if we merge back the modifications from trunk to the OSGI branch ver= y > often, it's never a good idea for a branch to last too long=E2=80=A6 It u= sually > make the last merge (when merging back to trunk) a nightmare. > With the component-hub working as standalone extendibility engine, there won't be major changes in the code i guess. But OSGI-fication of ApacheDS is some kind a incremental work, we'll face the problem and solve it, so i can't accurately say when the ApacheDS will run on OSGI completely. I hope the branch won't last longer than 2 month, but it's strongly needed right now even if ApacheDS runs on OSGI tomorrow :) > > Regards, > Pierre-Arnaud > > [1] - http://cl.ly/C3eB > [2] - http://cl.ly/C3O s Regards, G=C3=B6kt=C3=BCrk --bcaec5215ddd0a5da004b26986fd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Nov 23, 2011 at 5:31 PM, Pierre-= Arnaud Marcelot <pa= @marcelot.net> wrote:
Hi guys,

I wanted to give you a little bit of feedback and ask for status from the o= ther team members (especially G=C3=B6kt=C3=BCk) working on the OSGI branch.=


On the API side, G=C3=B6kt=C3=BCk introduced extensibility for the comparat= ors and syntax checkers using Apache Felix iPojo.
>From what I've seen it works great both as OSGI bundles and plain old j= ars.
=C2=A0
That's cool.=C2=A0



On the Studio side, I'm still struggling with these modifications.
Despite all my efforts and hacks (especially with the 'Bundle-Activatio= nPolicy' directive of the MANIFEST.MF file), I'm still unable to lo= ad extensible elements of the API within Studio.

Here's a quick recap of the problems I encountered.

At first, I noticed that in Equinox (Eclipse's OSGI container), the iPo= jo bundle and Shared's iPojo utilities bundle were not started and thei= r activators not called when I was using the API

I rebranded the iPojo bundle in Studio's projects to include the 'B= undle-ActivationPolicy' directive in its MANIFEST.MF file, and also inc= luded it in the 'ipojo-manager' of ApacheDS.

This is already a bad hack, but it is still not enough=E2=80=A6

Now, both bundles are started at the first time one of their classes is acc= essed by another bundle, and their activators are getting called correctly.=

However I'm still getting an exception, a NullPointerException.
This NPE occurs when we ask the bundle context for the service references r= elated to iPojo.
Here's a file containing the full stack trace of the exception: [1]

At this point, there should be a service reference for iPojo ready to use, = but we get nothing.

This is how far I've managed to go=E2=80=A6 It is not enough, but I'= ;ve tried every fix idea I had and I empty now=E2=80=A6
If someone has a clever idea=E2=80=A6 ;)
=C2=A0
<= div>I just have ideas on why this is failing. The first thing to consider i= s, something in eclipse equinox is preventing IPojo from actually initializ= ing. I mean it seems started but, actually it isn't. IPojo does not hav= e custom equinox console command, so that makes finding out what happened h= arder. If we assume IPojo is initialized correctly, then the problem is abo= ut the thread model of the equinox which leads Ipojo annotated api elements= (Comprators, etc...) to not registered with IPojo before the code looks for= them through IPojo. Starting them right after the IPojo in equinox may sol= ve the problem.

Actually there is no nice-looking solution to this prob= lem, because we're deploying osgi partially into the code, so we must m= anage these problems somehow. I couldn't have a chance to look at what = is wrong with the studio. IPojo runs on equinox for sure. I'll try to s= et my environment for debugging that problem.



On the ApacheDS side, I am able to run a small part of ApacheDS using the &= #39;apache-felix' module and its Eclipse launch configuration.

At the end of the execution I hit this exception:
org.apache.directory.server.config.ConfigurationException: Cannot initializ= e the org.apache.directory.server.core.normalization.NormalizationIntercept= or, error : java.lang.ClassNotFoundException: org.apache.directory.server.c= ore.normalization.NormalizationInterceptor not found by org.apache.director= y.server.service.builder [44]
Here's a file containing the full stack trace of the exception: [2]

I think it is, at the moment, the intended behavior since a few things stil= l remain to be set in ApacheDS to allow it to run successfully within an OS= GI container.

G=C3=B6kt=C3=BCk, can you confirm this is the intended behavior please?
=
=C2=A0
Yeah, that's the same stacktrace wit= h the one i get . Interceptors are not even visible to service-osgi bundle,= so NoClassDefFound is expected exception when we try to classload them whi= ch is what we do right now.=C2=A0

More importantly, can you share with us the status of the OSGI-fication of = ApacheDS? What is the purpose of the 'component-hub' you have been = working on lately?

component-hub is the= main extendibility layer i'm working on. This will provide us as devel= opers a facility to provide extendibility to any aspect of the ApacheDS. It= 's almost done with the implementation. It needs testing after that of = course. It will be a standalone engine in ApacheDS, means you won't hav= e to use OSGI directly to provide extendibility. ComponentHub provides faci= lities for, listening some type of components, creating instance of them, c= onfiguring the instances through associated entries in DIT, saving and load= ing those created/configured instances through java property files.

So when we want to provide extendibility for intercepto= rs lets say, we will write 3 class(schema generator, instance generator and= listener). Schema generator is for generating custom schema elements to re= present the component's instances on DIT. Instance generator in the oth= er hand must be compatible with its schema generator, and its purpose is ge= nerating instances with the default configuration which we'll extract f= rom the implementing class. When we register schema generator and instance = generator with the component-hub, our listener will start getting informati= on of all the interceptors in the system through callback methods with ADSC= omponent argument. Through this ADSComponent reference we'll be able to= create manage,create.configure the component instances, easilly. Each crea= ted instance will also be represented by ADSComponentInstance reference whi= ch will provide facilities for hooking its configuration with the some entr= y on DIT and so..I already wrote general purpose schema-instance generators= , but individual component types may need some extra, hardcoded things in i= ts schema and DIT entries, so that's why i make generators extendible. = I'll document the infrastructure after i finish and test it completely.= All i can say is after it finishes, making something in ApacheDS extendibl= e in all aspects will be really easy procedure.
=C2=A0


I hope we can move forward and find fixes to all the remaining issues quick= ly on OSGI.
Even if we merge back the modifications from trunk to the OSGI branch very = often, it's never a good idea for a branch to last too long=E2=80=A6 It= usually make the last merge (when merging back to trunk) a nightmare.
<= /blockquote>

With the component-hub working as standalone extendibil= ity engine, there won't be major changes in the code i guess. But OSGI-= fication of ApacheDS is some kind a incremental work, we'll face the pr= oblem and solve it, so i can't accurately say when the ApacheDS will ru= n on OSGI completely. I hope the branch won't last longer than 2 month,= but it's strongly needed right now even if ApacheDS runs on OSGI tomor= row :)
=C2=A0=C2=A0

Regards,
Pierre-Arnaud

[1] - http://cl.ly/C3eB=
[2] - http://cl.ly/C3O=C2= =A0s

Regards,
G=C3=B6kt=C3=BCrk --bcaec5215ddd0a5da004b26986fd--