Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of akarasulu@gmail.com
 designates 74.125.46.29 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=cmJZxEbv01FCRlRGGT3BdGRANJIM7K8IiwKaqOIK3GsjUbYnV0zfPN9gO/CCxEfS02
         WMriBy2oz0ufJKyb8Z+Ajb1AeHcqVCiK41oS8DYjYNIbkUEnPaLMdT8ZkVilacKYE1H3
         35CKq0G9R6Sp7OrNh9HaYb+7WpqQouPvqoPD0=
MIME-Version: 1.0
In-Reply-To: <49A9C767.5030101@nextury.com>
References: <49A9C767.5030101@nextury.com>
Date: Mon, 2 Mar 2009 14:07:20 -0500
Message-ID: <a32f6b020903021107g79778f04sa4c0ba5048b060d6@mail.gmail.com>
Subject: Re: Journal first draft
From: Alex Karasulu <akarasulu@gmail.com>
To: Apache Directory Developers List <dev@directory.apache.org>
Content-Type: multipart/alternative; boundary=000e0cd487ba2611160464278740

--000e0cd487ba2611160464278740
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi Emm,

On Sat, Feb 28, 2009 at 6:23 PM, Emmanuel Lecharny <elecharny@apache.org>wrote:

> Hi guys,
>
> I'm currently working on a very preliminary implementation of the journal.
> It is an interceptor added at the very end of the chain, just before the
> partition. the way it works is very simple :
> - it logs the LDIF for every modification in the journal, plus some extra
> informations (Ldapprincipal, a timestamp and a revision number). This is
> done before calling the partition Add method
> - when the entry has been added into the partition, an ACK is logged into
> the journal, as a comment, containing the revision of the ACKed operation.
> - if the addition failed, a NACK is logged in the same way
> - the written information are immediately flushed on disk, and the write
> operation is synchronized in order to avoid a mixup of operations in the
> file.
>
> Here is an exemple of what we get :
>
> # 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system
> # ts: 1235862542574
> # rev: 1235862541942
> dn: 2.5.4.3=kate#bush,2.5.4.11=system
> changeType: Add
> createtimestamp: 20090228230902Z
> sn: Bush
> entryuuid:: OWM5MDU5YjYtNmJkNC00ZmQzLTgwODMtMzE4MGJhOGQyMGY0
> cn: Kate#Bush
> entrycsn:: MjAwOTAzMDEwMDA5MDIuMDAwNTM0WiMwIzAjMDAwMDAw
> objectclass: person
> objectclass: top
> creatorsname: 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system
>
> # ack-rev: 1235862541942
>
> # 0.9.2342.19200300.100.1.1=admin,2.5.4.11=system
> # ts: 1235862542618
> # rev: 1235862541943
> dn: 2.5.4.3=kate#bush,2.5.4.11=system
> changeType: Delete
>
> # ack-rev: 1235862541943
> ...
>
> The revision number is used to associate a ack/nack to an operation : if
> the server crashes in the middle of a partition update, we won't have a
> ack/nack for the pending operation.
>
> As you can see, this is a very basic implementation atm. I didn't dealt
> with all the intricacy of journal rotation, journal cleanup, and such. It's
> a growing file, and it can grow fast. Some immediate improvement could be to
> use more than one file to store the changes, with an executor to deal with
> the write, allowing parallel update of the journal (it would be a bit more
> complicated to restore the journal, but at least, we would avoid a
> contention problem).
>
> So wdyt about this first version ? What's missing ?
>

(1) Why not make these comment attribute actual attributes in the LDIF even
though they are special?

(2) Why is rev not a regular attribute?

(3) What exactly is the difference between ACK and NACK and why do we need a
NACK?

Thanks,
Alex

--000e0cd487ba2611160464278740
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Emm,<br><br><div class=3D"gmail_quote">On Sat, Feb 28, 2009 at 6:23 PM, =
Emmanuel Lecharny <span dir=3D"ltr">&lt;<a href=3D"mailto:elecharny@apache.=
org">elecharny@apache.org</a>&gt;</span> wrote:<br><blockquote class=3D"gma=
il_quote" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0=
pt 0pt 0.8ex; padding-left: 1ex;">
Hi guys,<br>
<br>
I&#39;m currently working on a very preliminary implementation of the journ=
al. It is an interceptor added at the very end of the chain, just before th=
e partition. the way it works is very simple :<br>
- it logs the LDIF for every modification in the journal, plus some extra i=
nformations (Ldapprincipal, a timestamp and a revision number). This is don=
e before calling the partition Add method<br>
- when the entry has been added into the partition, an ACK is logged into t=
he journal, as a comment, containing the revision of the ACKed operation.<b=
r>
- if the addition failed, a NACK is logged in the same way<br>
- the written information are immediately flushed on disk, and the write op=
eration is synchronized in order to avoid a mixup of operations in the file=
.<br>
<br>
Here is an exemple of what we get :<br>
<br>
# 0.9.2342.19200300.100.1.1=3Dadmin,2.5.4.11=3Dsystem<br>
# ts: 1235862542574<br>
# rev: 1235862541942<br>
dn: 2.5.4.3=3Dkate#bush,2.5.4.11=3Dsystem<br>
changeType: Add<br>
createtimestamp: 20090228230902Z<br>
sn: Bush<br>
entryuuid:: OWM5MDU5YjYtNmJkNC00ZmQzLTgwODMtMzE4MGJhOGQyMGY0<br>
cn: Kate#Bush<br>
entrycsn:: MjAwOTAzMDEwMDA5MDIuMDAwNTM0WiMwIzAjMDAwMDAw<br>
objectclass: person<br>
objectclass: top<br>
creatorsname: 0.9.2342.19200300.100.1.1=3Dadmin,2.5.4.11=3Dsystem<br>
<br>
# ack-rev: 1235862541942<br>
<br>
# 0.9.2342.19200300.100.1.1=3Dadmin,2.5.4.11=3Dsystem<br>
# ts: 1235862542618<br>
# rev: 1235862541943<br>
dn: 2.5.4.3=3Dkate#bush,2.5.4.11=3Dsystem<br>
changeType: Delete<br>
<br>
# ack-rev: 1235862541943<br>
...<br>
<br>
The revision number is used to associate a ack/nack to an operation : if th=
e server crashes in the middle of a partition update, we won&#39;t have a a=
ck/nack for the pending operation.<br>
<br>
As you can see, this is a very basic implementation atm. I didn&#39;t dealt=
 with all the intricacy of journal rotation, journal cleanup, and such. It&=
#39;s a growing file, and it can grow fast. Some immediate improvement coul=
d be to use more than one file to store the changes, with an executor to de=
al with the write, allowing parallel update of the journal (it would be a b=
it more complicated to restore the journal, but at least, we would avoid a =
contention problem).<br>

<br>
So wdyt about this first version ? What&#39;s missing ?<br>
</blockquote></div><br>(1) Why not make these comment attribute actual attr=
ibutes in the LDIF even though they are special? <br><br>(2) Why is rev not=
 a regular attribute?<br><br>(3) What exactly is the difference between ACK=
 and NACK and why do we need a NACK?<br>
<br>Thanks,<br>Alex<br>

--000e0cd487ba2611160464278740--