directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel L├ęcharny <>
Subject Replication internal design proposal
Date Mon, 03 Dec 2012 13:28:22 GMT
Hi guys,

reviewing the current replication implementation, I have a few proposals
to improve some overhead.

First, let's describe the way it works :

1) LdapServer starts the consumers
  for each of them, we create a new thread that init the consumer, and
start it (consumer.start())

2) We try to connect to the provider, in a loop. When we get connected,
we call startSync() that will start the replication

3) In startSync, we handle two different cases :
 - refreshOnly uses a thread that loop until stopped (which occurs only
when we shutdown the LdapServer)

Here, we have an issue : we don't handle the disconnection from a remote

There is also no need to create a new thread, we can use the one that
has been created while starting the consumer

 - refreshAndPersist calls the doSyncSearch() method which does a
search, refresh the data if it can, and wait for any modification done
on the provider
If the refresh can't succeed - for instance because the provider has not
enough information to restart from the starting point -, we will have to
resend a new search request but with an emtpy cookie This is currently
done using a recursive call (see my previous mail).
Otherwise, we loop until we get disconnected, or until the searchFuture
get invalidated, or until the provider returns a SearchResulDone.

We should not call the doSyncSearch() method recursively

When we exit the loop, we should exit the method with a status that will
be handled by the main thread. It up to the caller to know what should
be done :
 - recalling the method immediately if a new refresh is needed, with an
emtpy cookie
 - wait for reconnection to be established again
 - wait for the configuration to be fixed (in case the pb was a
configuration issue)

When the provider get disconnected, the way it works is a bit subtil :
the connectionClosed() method is called, and restart the consumer.
Except that it starts it in another thread, because we may have exited
from the original thread. (the connectionClosed() call is done using a
thread from the MINA executor, and if the disconnection has occured, the
consumer is not anymore handled by the thread that started it). Unless
I'm missing something.

Here, I would suggest to *never* get out of the thread that started the
consumer, except if we explicitely require so. When the
connectionClosed() method is called, we simply exit from the loop we are
in, and go to sleep before retrying to reconnect later.

Thoughts ?

Emmanuel L├ęcharny 

View raw message