Here is an initial idea about implementation which I have in my mind
HOWL has a feature called 'marking' in its log file (a.k.a journal). The idea is to use this as a checkpoint since
the last successful disk write of the DIT data i.e whenever we perform a sync we put a mark in the journal.
in case of a crash we can retrieve the data from journal from the marked position(using howl API),
Currently the syncing of DIT data is a per partition based operation unless a call is made to
DirectoryService's sync() method which internally calls the sync on PartitionNexus.
IMO this marking of journal should happen in the DirectoryService's sync() operation.
A change to the partition's sync() method to call DirectoryService's sync() which intern calls (each) partition's
commit() (a new method) would help. A special flag like 'isDirty' in the partition will allow us to avoid calling
commit() on each partition.
Any other ideas about how best we can maintain a checkpoint/mark after *any* sync operation on DIT data?.
Having said this, I have another issue, how do I detect the beginning of a corrupted
entry in a JDBM file(all the DIT data is stored in these files)
To put this in other way, if a JDBM file was synced at nth entry and server was crashed in the middle of
writing n+1th entry I would like to start recovery from the end of nth record (a common idea I believe though)
(haven't looked at jdbm code yet, but throwing this question anticipating a quick answer ;) )