directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Howard Chu <...@symas.com>
Subject Re: [jira] Created: (DIRSERVER-1098) New replicas may never be synchronised
Date Thu, 15 Nov 2007 01:40:25 GMT
Emmanuel Lecharny wrote:
> Howard Chu wrote:
>> Is there a doc somewhere that describes the replication protocol?
> Here you are :
> http://cwiki.apache.org/confluence/display/DIRxSRVx11/Mitosis
> 
> These pages are really very rough.

Thanks. I see you've read through all the previous failures on this path...

>> We encountered similar issues when developing the multimaster support
>> in OpenLDAP. Our solution was simply to assign unique server IDs to
>> each master, and record the last known modification for each known
>> server ID. This way local modifications don't overwrite the (lack of)
>> state from other servers.
>>
>> Another issue to consider when bringing up a new replica - you may
>> want to disallow all queries completely, until the replica has fully
>> sync'd up. If you allow searches during this initial phase, a client
>> is likely to get a partial set of results. In some cases, that may be
>> worse than getting no result...

> The pb is that if you have a lot of replication, it may not be a viable
> approach. Also if you want to implement the 'paging control', as it's up
> to the client to request the next results, you have no good solutions...

> At some point, I think we should consider that a LDAP server is not like
> a RDBMS system with a full transaction system, and you must assume that
> dirty read is a standard mode. That does not mean we shouldn't consider
> blocking incoming requests until a sync-up is done when a server is
> restarted.

In OpenLDAP I've recommended that a replica chain all incoming requests to a 
provider while it's in startup mode. There can be transient inconsistencies at 
later points in its runtime, and we live with those. But if you're bringing up 
a brand new server, it's best to pretend it doesn't even exist until it's at 
least 80-90% sync'd. Obviously you can manage this outside of LDAP (e.g., if 
you have a DNS pool or hardware load-balancer directing clients to servers, 
don't add the new replica to the pool until it's ready) for a new server.

> This is not a simple problem...

Heh heh. So very true.

-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

Mime
View raw message