directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lecharny <elecha...@apache.org>
Subject Journal first draft
Date Sat, 28 Feb 2009 23:23:19 GMT
Hi guys,

I'm currently working on a very preliminary implementation of the 
journal. It is an interceptor added at the very end of the chain, just 
before the partition. the way it works is very simple :
- it logs the LDIF for every modification in the journal, plus some 
extra informations (Ldapprincipal, a timestamp and a revision number). 
This is done before calling the partition Add method
- when the entry has been added into the partition, an ACK is logged 
into the journal, as a comment, containing the revision of the ACKed 
operation.
- if the addition failed, a NACK is logged in the same way
- the written information are immediately flushed on disk, and the write 
operation is synchronized in order to avoid a mixup of operations in the 
file.

Here is an exemple of what we get :

# 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system
# ts: 1235862542574
# rev: 1235862541942
dn: 2.5.4.3=kate#bush,2.5.4.11=system
changeType: Add
createtimestamp: 20090228230902Z
sn: Bush
entryuuid:: OWM5MDU5YjYtNmJkNC00ZmQzLTgwODMtMzE4MGJhOGQyMGY0
cn: Kate#Bush
entrycsn:: MjAwOTAzMDEwMDA5MDIuMDAwNTM0WiMwIzAjMDAwMDAw
objectclass: person
objectclass: top
creatorsname: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system

# ack-rev: 1235862541942

# 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system
# ts: 1235862542618
# rev: 1235862541943
dn: 2.5.4.3=kate#bush,2.5.4.11=system
changeType: Delete

# ack-rev: 1235862541943
...

The revision number is used to associate a ack/nack to an operation : if 
the server crashes in the middle of a partition update, we won't have a 
ack/nack for the pending operation.

As you can see, this is a very basic implementation atm. I didn't dealt 
with all the intricacy of journal rotation, journal cleanup, and such. 
It's a growing file, and it can grow fast. Some immediate improvement 
could be to use more than one file to store the changes, with an 
executor to deal with the write, allowing parallel update of the journal 
(it would be a bit more complicated to restore the journal, but at 
least, we would avoid a contention problem).

So wdyt about this first version ? What's missing ?

-- 
--
cordialement, regards,
Emmanuel L├ęcharny
www.iktek.com
directory.apache.org



Mime
View raw message