directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <>
Subject Re: [DRS] Changelog improvements
Date Mon, 26 Jan 2009 16:55:50 GMT
A transaction journal (a.k.a. transaction log) is used to rapidly persist
write operations arriving at the server before being processed.  Local
transactions for these operations are opened to apply the change. This
allows the server to replay operations which did not complete due to some
interruption during processing. It also allows the server eventually to
apply changes and their side effects (like those from triggers), in the same
transaction which allows all to rollback together or to succeed together.
We need these things eventually even though they may not be critical right
now for replication.

Journal file formats are simple, with indices into these files to track
those operations that have completed from those that have not.  Journals are
not ideal for a search-able history retrieval system to be used for auditing
and snapshotting.  The history CL requires many more indices and it's
information needs to be more structured.  Conversely search-able CL is not
ideal as a transaction log since organizing the information and updating all
these indices requires disk operations which take too much time.

We have to be clear on what we want as a set of requirements.  If we're
going to implement a transaction journal/log here's what I'd like to see:

0). Very fast write of operation information to disk including any
information needed to rollback an operation.
1). The journal should be the basis for implementing local transactions and
indices into it should be minimized for performance sake.
2). The transaction log should be pruned asynchronously removing operations
that have been processed.  These operations can then be pumped into the CL,
for audit history and snapshotting.

WDYT? Please add more, I have more thoughts which I can add later.


On Mon, Jan 26, 2009 at 11:29 AM, Emmanuel Lecharny <>wrote:

> >> Just because I think it's easier to recover from a crashed LDIF file
> >> than from a crashed JDBM file :)
> >
> > Good point.  However as you said we might want a separate journal using
> > something howl or the alternative that David Jencks pointed out being
> using
> > by the AMQ folks.
> Well, we should not close doors at this point. I know nothing about
> howl, so I would like to see if we can use it.

View raw message