directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kiran Ayyagari <kayyag...@apache.org>
Subject Re: Replication internal design proposal
Date Mon, 03 Dec 2012 14:59:02 GMT
On Mon, Dec 3, 2012 at 6:58 PM, Emmanuel L├ęcharny <elecharny@gmail.com>wrote:

> Hi guys,
>
> reviewing the current replication implementation, I have a few proposals
> to improve some overhead.
>
> First, let's describe the way it works :
>
> 1) LdapServer starts the consumers
>   for each of them, we create a new thread that init the consumer, and
> start it (consumer.start())
>
> 2) We try to connect to the provider, in a loop. When we get connected,
> we call startSync() that will start the replication
>
> 3) In startSync, we handle two different cases :
>  - refreshOnly uses a thread that loop until stopped (which occurs only
> when we shutdown the LdapServer)
>
> <note>
> Here, we have an issue : we don't handle the disconnection from a remote
> provider.
> </note>
>
> no we do, note that the connectionClosed() is called by the
LdapNetworkConnection based on MINA's sessionClosed() callback

> <note>
> There is also no need to create a new thread, we can use the one that
> has been created while starting the consumer
> </note>
>
> no, we don't create a new thread, the thread is created once and it stays
alive till the server stops

>  - refreshAndPersist calls the doSyncSearch() method which does a
> search, refresh the data if it can, and wait for any modification done
> on the provider
> If the refresh can't succeed - for instance because the provider has not
> enough information to restart from the starting point -, we will have to
> resend a new search request but with an emtpy cookie This is currently
> done using a recursive call (see my previous mail).
> Otherwise, we loop until we get disconnected, or until the searchFuture
> get invalidated, or until the provider returns a SearchResulDone.
>
> <note>
> We should not call the doSyncSearch() method recursively
> </note>
>
> +1

> <note>
> When we exit the loop, we should exit the method with a status that will
> be handled by the main thread. It up to the caller to know what should
> be done :
>  - recalling the method immediately if a new refresh is needed, with an
> emtpy cookie
>  - wait for reconnection to be established again
>  - wait for the configuration to be fixed (in case the pb was a
> configuration issue)
> </note>
>
> +1

> When the provider get disconnected, the way it works is a bit subtil :
> the connectionClosed() method is called, and restart the consumer.
> Except that it starts it in another thread, because we may have exited
> from the original thread. (the connectionClosed() call is done using a
> thread from the MINA executor, and if the disconnection has occured, the
> consumer is not anymore handled by the thread that started it). Unless
> I'm missing something.
>
> <note>
> Here, I would suggest to *never* get out of the thread that started the
> consumer, except if we explicitely require so. When the
> connectionClosed() method is called, we simply exit from the loop we are
> in, and go to sleep before retrying to reconnect later.
> </note>
>
> again, we don't create/start a new thread, the same thread is used

> Thoughts ?
>
> --
> Regards,
> Cordialement,
> Emmanuel L├ęcharny
> www.iktek.com
>
>


-- 
Kiran Ayyagari
http://keydap.com

Mime
View raw message