directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kiran Ayyagari <ayyagariki...@gmail.com>
Subject [DRS] thoughts about implementation
Date Sun, 01 Feb 2009 18:49:23 GMT
Hello guys,

     Here is an initial idea about implementation which I have in my mind

     HOWL has a feature called 'marking' in its log file (a.k.a journal). The idea is to use
this as a checkpoint since
     the last successful disk write of the DIT data i.e whenever we perform a sync we put
a mark in the journal.
     in case of a crash we can retrieve the data from journal from the marked position(using
howl API),

     Currently the syncing of DIT data is a per partition based operation unless a call is
made to
     DirectoryService's sync() method which internally calls the sync on PartitionNexus.

     IMO this marking of journal should happen in the DirectoryService's sync() operation.

     A change to the partition's sync() method to call DirectoryService's sync() which intern
calls (each) partition's
     commit() (a new method) would help. A special flag like 'isDirty' in the partition will
allow us to avoid calling
     commit() on each partition.

     Any other ideas about how best we can maintain a checkpoint/mark after *any* sync operation
on DIT data?.

     Having said this, I have another issue, how do I detect the beginning of a corrupted
     entry in a JDBM file(all the DIT data is stored in these files)

     To put this in other way, if a JDBM file was synced at nth entry and server was crashed
in the middle of
     writing n+1th entry I would like to start recovery from the end of nth record (a common
idea I believe though)
     (haven't looked at jdbm code yet, but throwing this question anticipating a quick answer
;) )

     One idea comes to my mind as I write this is to read each entry from the beginning of
the jdbm file using the
     serializer classes we provide for storing the pair (K,V)

     thoughts?

-- 
Kiran Ayyagari

Mime
View raw message