On Mon, Nov 28, 2011 at 11:56 AM, Alex Karasulu <akarasulu@apache.org> wrote:
On Mon, Nov 28, 2011 at 2:25 AM, Göktürk Gezer <gokturk@apache.org> wrote:
> Hi Devs,
> As i go through implementing component-hub, i couldn't think clearly about
> some things, so i need some pointers on them. I've made some decisions on
> code based on my experiments but i need to hear more from you on these
> specific issues.
> component-hub is responsible for providing main extendibility facilities,
> which contains the DIT configuration hooks. component-hub is providing the
> DIT configuration hooks as a way for users to configure the component
> instances.( the instance of a EventInterceptor for example). Then some
> DirectoryListener implementation will listen for specified DIT entries for
> changes and then reconfigure the instances. First question is whether should
> i use PersistentSearch or implement DirectoryListener interface myself.
> DirectoryListeners are called by EventInterceptor so that will make the
> listening code have to go through LdapCoreSession. How PersistentSearch is
> different? Is there any mechanism other than these two??

OBJECTIVE: Detect changes to LDAP backed component configurations and
react to them.

PersistentSearch is a protocol mechanism (over the wire) used to
detect changes to entries (in remote LDAP servers) implemented using
controls. You issue a search request with the persistent search
control and specifying an area of the DIT to watch. The search never
completes unless you abort it and it just sends you back entries if
they change with a control indicting the nature of the change. It's a
hack if you ask me. So this mechanism can work for remote and local
servers but you'll go through the wire, the protocol, and will pay a
big price for it from the number of threads used to encoding/decoding
etc etc.

Another way to detect changes but only to a local (in the same process
space) DIT, is by using the event listener mechanism provided by the
Event subsystem of the DirectoryService. This does essentially the
same thing that persistent search does for the same local directory
service but better and with less overhead. "Better", in that it allows
you to see exactly what attributes changed and how which is a tiny bit
more resolution than from the response control you get back with the
persistent search mechanism.


1). You're local to the data anyway, configuring local components
2). Less overhead, more efficient faster ... yada yada
3). It's a better programatic interface.
4). Does everything you need it to do.

This of course is for the solid state of configuration in DIT
operation after the server has already bootstrapped and is online. To
actually bootstrap the server you'll probably need to read data kept
in LDIF files in the configuration partition folder on disk as if it's
stored in an LDAP server or a live partition but there is no LDAP
server yet, nor is there a live partition yet.

Makes sense?

Okey then, i just wondered what PersistentSearch is meant to. compenent-hub uses event mechanism already. I'm leaving it like that.

> Second thing is like chicken-and-egg problem, specifically about
> Interceptors. To add the custom generated schemas and the DIT configuration
> hooks for instances, the current code access the server through session
> (LdapCoreSessionConnection). But for LdapCoreSession to work as expected,
> the interceptors are needed(at least, to work in the way it meant to). How i
> solved this issue is by deferring all the write operations until
> DirectoryService is fully initialized. component-hub will be able to access,
> instantiate and use the interceptors( and other types of components).
> However, until its being notified of full initialization of
> DirectoryService, it defers the custom schema and DIT configuration hook
> entries to be written to DIT. Is there any problem with that approach that i
> couldn't see? This approach is clear but it may introduce some rendezvous
> like concurrency management into code, increasing server startup about 1
> second. The other way to do is initializing component-hub with the
> DirectoryService reference having its Schema and Config Partition set. And
> then we access the partitions directly without going through nexus and
> interceptors. But to handle it with the second way, we must be sure nothing
> is dependent on Interceptors until server is fully initialized.(*1). Which
> way would you choose?

I'd implement an LDIF file reading LdapConnection that is just
READ-ONLY with just simple operations enabled to read what we need
from the configuration at startup. Then you can use this object at
initialization/bootstrap time to get at the configuration data you
need in the LDIF files stored on disk. Then there is no chicken and
egg problem. Then after bootstrapping you just throw away this
LdifFileLdapConnection or whatever you want to call it and can use the
session connections instead for steady solid state operation after

I couldn't get my answer on that actually. First of all, the configuration information for component-hub and individual components are not all in ldif format. All the configurations end up being an entry in DIT but i thought saving the configuration like that is not feasible for many reasons. We have to do many housekeeping to leave DIT clear, after every OSGI operation if we go that way. That's why configuration is kept per component:
* When we uninstall an individual component from the OSGI framework, all of its schema and configuration entries in DIT are cached, leaving DIT as this component has never been installed. 
* When we install that component later time, its cache is stored in schema and config partitions.

Schema caches are kept as ldif files, but configuration caches are kept as java property files. This is gives component-hub an ability to fetch the configuration but configuration alone, and then hook it somewhere in the DIT dynamically. This caching system is designed for server runtime in focus, but its making server startups-shutdowns bulky.

But in the time i'm writing this, i realized this system is not resilliant for disaster scenarios. I must change it to load configurations from server's config partition itself and do some housekeeping to make DIT reflect the current status of the OSGI container as much as it can. I like it when ideas come to your mind in the middle of writing an email :)

> (*1) init() method of SubEntryInterceptor is using nexus to do some search.
> But at that point in the code no partition is added to the nexus, not even
> schema partition is set on DirectoryService. What exactly the code there is
> doing, is it broken or am i missing something?

Sorry don't have the code up in your branch. Let me set that up and I
can be more useful to you.

Its on the trunk also.

Best Regards,
-- Alex