directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Howard Chu <...@symas.com>
Subject Re: [ApacheDS][Mitosis] Replication data
Date Thu, 06 Dec 2007 00:22:49 GMT
Martin Alderson wrote:
> Thanks for the responses, all.
> 
> Apologies for the delay in getting back to you - having a family problem
> at the moment so have very little spare time.
> 
> I thought having the replication logs stored in LDAP sounded nice - for
> new replicas we have to send all replicatable entries but after that the
> log LDAP entries can be sent instead.  It would be pretty much the same
> code logic and it just seemed to solve all the problems with a large
> amount of code re-use.  I was worried about possible performance hits
> though and it sounds like you (Alex) don't want to store the logs in
> LDAP for the same reason.

That all makes sense. That's the rationale behind the OpenLDAP accesslog 
overlay, which implements our logging schema.

> My main reasons for suggesting storing the logs in LDAP are:
> 1. So we can have optional attributes in each log entry.  This is needed
> when we "explode" the current message blob so it can be queried
> efficiently.  With JDBM I guess we would have to specify a new table for
> each type of message.

The logging schema format we use is already efficiently searchable.

> 2. To reduce the code complexity.  We would have virtually the same code
> for sending whole entries as sending the logs and we would have less
> code for dealing with the data storage in general.
> 3. To reduce the current tight coupling with the backend database.  By
> using LDAP as the abstraction layer we could leverage ApacheDS' existing
> mechanism for specifying the data store.
> 4. To allow an easy way to view the logs.
> 5. It seems to be the most natural fit.  Since we need to store (part
> of) an LDAP entry in the logs, why not store it in LDAP?

> I'll take another stab at explaining that: we already have code to store
> LDAP entries in a database, so why would we want to write that again?

Yes to all of the above.

>  >> The biggest concern I have for this is the inflexibility of LDAP
>  >> searches. Do we have a sort control in ApacheDS?
>  > What types of searches do you envision performing, for which LDAP
>  > is too inflexible? OpenLDAP's syncrepl can be pretty much entirely
>  > mapped onto plain search operations. We gain a lot of versatility
>  > by keeping things generic.
> 
> We need to search for log entries beyond a certain CSN and have the
> results ordered based on CSN.  I guess if the results are always
> returned in creation date order then it might not be an issue (I'm not
> yet sure what ApacheDS does or what the LDAP standard says).

As Kurt used to remind me, a CSN is a Change Sequence Number, but it is not a 
Commit Sequence Number. The order in which you see CSNs isn't necessarily the 
order in which those changes were committed in the DB. As such, the syncrepl 
protocol assumes that the changes it receives are in random order. That means 
we can't update our replication state upon every entry received. So 
incremental refreshes can only update the replica state once the refresh has 
completed, at which point we record the greatest CSN we saw. (This also means 
that if a refresh is interrupted, it starts over from the exact same starting 
point on the next retry. The syncrepl consumer does comparisons to discard any 
received updates that it has seen before.)

> Currently
> we also find the current CSN vector by just getting the most recent log
> - we do this by performing a search with inverted sort by CSN and 1
> result maximum.

Hm, we only search for that at startup time; at runtime it's always maintained 
in memory.

> Also, if we have the attributes in a child entry of the
> actual log entry as I suggested we would need to specify a parent-child
> relationship in the search.

That sounds like a painful model to implement.

>  >> Also our MMR support is still immature, we don't yet do value-level
>  >> conflict resolution.
>  > Yeash we have yet to consider that.
> 
> We will have this once I have fixed
> https://issues.apache.org/jira/browse/DIRSERVER-894.
> 
> 
>  > The trick to get from basic single-master to basic (entry-level
>  > only) multi-master is just to store multiple contextCSNs - one for
>  > each peer master, and ignore entry updates that are older than an
>  > entry's current entryCSN. The other requirement here is that you
>  > have reliable, tightly synced clocks, otherwise the conflict
>  > resolution policy falls apart.
> 
> That's exactly how our replication module works at the moment except we
> just send the changes rather than the whole entry.  I am currently
> looking at improving the way we store the logs so we can efficiently do
> attribute value level conflict resolution.  I suspect that I will end up
> with something very similar to delta-syncrepl.  I will try and dig out
> some information on that from the openldap mailing list.

Yeah, the stuff Emmanuel pointed me at indicated that your approach is already 
very similar to syncrepl.

-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

Mime
View raw message