directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selcuk AYA <ayasel...@gmail.com>
Subject Re: Questions about the design of the "component-hub"
Date Tue, 29 Nov 2011 11:07:15 GMT
On Tue, Nov 29, 2011 at 12:49 PM, Emmanuel Lécharny
<elecharny@apache.org> wrote:
> On 11/29/11 11:28 AM, Alex Karasulu wrote:
>>
>> On Tue, Nov 29, 2011 at 11:59 AM, Emmanuel Lecharny<elecharny@gmail.com>
>>  wrote:
>>>
>>> On 11/28/11 10:56 AM, Alex Karasulu wrote:
>>>>
>>>> On Mon, Nov 28, 2011 at 2:25 AM, Göktürk Gezer<gokturk@apache.org>
>>>>  wrote:
>>>>>
>>>>> Hi Devs,
>>>>> As i go through implementing component-hub, i couldn't think clearly
>>>>> about
>>>>> some things, so i need some pointers on them. I've made some decisions
>>>>> on
>>>>> code based on my experiments but i need to hear more from you on these
>>>>> specific issues.
>>>>> component-hub is responsible for providing main extendibility
>>>>> facilities,
>>>>> which contains the DIT configuration hooks. component-hub is providing
>>>>> the
>>>>> DIT configuration hooks as a way for users to configure the component
>>>>> instances.( the instance of a EventInterceptor for example). Then some
>>>>> DirectoryListener implementation will listen for specified DIT entries
>>>>> for
>>>>> changes and then reconfigure the instances. First question is whether
>>>>> should
>>>>> i use PersistentSearch or implement DirectoryListener interface myself.
>>>>> DirectoryListeners are called by EventInterceptor so that will make the
>>>>> listening code have to go through LdapCoreSession. How PersistentSearch
>>>>> is
>>>>> different? Is there any mechanism other than these two??
>>>>
>>>> OBJECTIVE: Detect changes to LDAP backed component configurations and
>>>> react to them.
>>>>
>>>> PersistentSearch is a protocol mechanism (over the wire) used to
>>>> detect changes to entries (in remote LDAP servers) implemented using
>>>> controls. You issue a search request with the persistent search
>>>> control and specifying an area of the DIT to watch. The search never
>>>> completes unless you abort it and it just sends you back entries if
>>>> they change with a control indicting the nature of the change. It's a
>>>> hack if you ask me. So this mechanism can work for remote and local
>>>> servers but you'll go through the wire, the protocol, and will pay a
>>>> big price for it from the number of threads used to encoding/decoding
>>>> etc etc.
>>>>
>>>> Another way to detect changes but only to a local (in the same process
>>>> space) DIT, is by using the event listener mechanism provided by the
>>>> Event subsystem of the DirectoryService. This does essentially the
>>>> same thing that persistent search does for the same local directory
>>>> service but better and with less overhead. "Better", in that it allows
>>>> you to see exactly what attributes changed and how which is a tiny bit
>>>> more resolution than from the response control you get back with the
>>>> persistent search mechanism.
>>>>
>>>> SO YOU SHOULD MOST LIKELY USE THE EVENT MECHANISM. This because:
>>>>
>>>> 1). You're local to the data anyway, configuring local components
>>>> 2). Less overhead, more efficient faster ... yada yada
>>>> 3). It's a better programatic interface.
>>>> 4). Does everything you need it to do.
>>>>
>>>> This of course is for the solid state of configuration in DIT
>>>> operation after the server has already bootstrapped and is online.
>>>
>>> +1. This is definitively the way to go.
>>>>
>>>> To
>>>> actually bootstrap the server you'll probably need to read data kept
>>>> in LDIF files in the configuration partition folder on disk as if it's
>>>> stored in an LDAP server or a live partition but there is no LDAP
>>>> server yet, nor is there a live partition yet.
>>>
>>> The Config is read *before* the DS is started. This is actually the very
>>> first thing we do. You must assume the configuration partition is
>>> available,
>>> and it's browsable. You don't need the LDAP server or the DS to access
>>> the
>>> information stored there, as we have it managed by an AvlPartition (with
>>> its
>>> own searchengine and all what is needed).
>>>
>>>> Makes sense?
>>>>
>>>>
>>>>> Second thing is like chicken-and-egg problem, specifically about
>>>>> Interceptors. To add the custom generated schemas and the DIT
>>>>> configuration
>>>>> hooks for instances, the current code access the server through session
>>>>> (LdapCoreSessionConnection). But for LdapCoreSession to work as
>>>>> expected,
>>>>> the interceptors are needed(at least, to work in the way it meant to).
>>>>> How i
>>>>> solved this issue is by deferring all the write operations until
>>>>> DirectoryService is fully initialized. component-hub will be able to
>>>>> access,
>>>>> instantiate and use the interceptors( and other types of components).
>>>>> However, until its being notified of full initialization of
>>>>> DirectoryService, it defers the custom schema and DIT configuration
>>>>> hook
>>>>> entries to be written to DIT. Is there any problem with that approach
>>>>> that i
>>>>> couldn't see? This approach is clear but it may introduce some
>>>>> rendezvous
>>>>> like concurrency management into code, increasing server startup about
>>>>> 1
>>>>> second. The other way to do is initializing component-hub with the
>>>>> DirectoryService reference having its Schema and Config Partition set.
>>>>> And
>>>>> then we access the partitions directly without going through nexus and
>>>>> interceptors. But to handle it with the second way, we must be sure
>>>>> nothing
>>>>> is dependent on Interceptors until server is fully initialized.(*1).
>>>>> Which
>>>>> way would you choose?
>>>>
>>>> I'd implement an LDIF file reading LdapConnection that is just
>>>> READ-ONLY with just simple operations enabled to read what we need
>>>> from the configuration at startup.
>>>
>>> We already have it. This is the way the config in dit is handled.
>>>>
>>>>  Then you can use this object at
>>>> initialization/bootstrap time to get at the configuration data you
>>>> need in the LDIF files stored on disk. Then there is no chicken and
>>>> egg problem. Then after bootstrapping you just throw away this
>>>> LdifFileLdapConnection or whatever you want to call it and can use the
>>>> session connections instead for steady solid state operation after
>>>> startup.
>>>
>>> Keep in mind that since 2.0.0-M1, we use the configuration in DIT, which
>>> is
>>> read before we initialize anything, as we need to read the configuration
>>> to
>>> be able to "cofigure" the server. This solve the chicken/egg issue.
>>
>> That solves a great deal of problems. So the configuration partition
>> is setup and available for reads without having to setup any other
>> ApacheDS machinery?
>
> From the top of my head, yes. As every little configurable part of ADS is
> configured with what has been stored in the DIT, it's totally natural.
>
>>
>> Is the raw partition used for access? How is this going to change with
>> some of the movements done by Selcuk to move search out of the
>> partition into an upper layer?
>
> We read the LDIF partition from disk, then we load the configuration
> elements as POJOs. We use reflection to do that.
>
> Your second point is a good question. But afaiu, it's not because the search
> is extracted from the partition code that we can't use it without any other
> elements from the DS machinery. Or at least, we should be able to use it
> independently.
>
> This may need to be discussed.

Ldif loading and parsing at initialization works independently of the
search engine. To make this more sane, I made ldif partitions write
their data to a shadow file and do a move to the ldif file during
sync.

>
> PS : can you 'reply list' instead of 'reply all' ? Every time you reply all,
> I'm forced to select the list instead of your single name, as you as the
> recipient is automatically selected as if you sent me a personal mail.
> Thanks !
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>

Mime
View raw message