directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel L├ęcharny <elecha...@gmail.com>
Subject Database corruption, some interesting finding !
Date Mon, 25 Jan 2016 11:21:33 GMT
Hi guys,

while working on the repair() function, iI found something very
interesting.

When we update a partition, at the end, we call the sync() method which
writes down on disk all the updates which are pending in memory. This
sync() method itself sync all the indexes :

    /**
     * This method is called when the synch thread is waking up, to write
     * the modified data.
     *
     * @throws Exception on failures to sync database files to disk
     */
    public synchronized void sync() throws Exception
    {
        if ( !initialized )
        {
            return;
        }

        // Sync all system indices
        for ( Index<?, String> idx : systemIndices.values() )
        {
            idx.sync();
        }

        // Sync all user defined userIndices
        for ( Index<?, String> idx : userIndices.values() )
        {
            idx.sync();
        }
       
        // Sync the master table
        ( ( JdbmMasterTable ) master ).sync();
    }

Sadly, the rdnIndex is *never* flushed explicitely - so it's done only
periodically (every 2000 updates, the default):

    /**
     * Commit the modification on disk
     *
     * @param recordManager The recordManager used for the commit
     */
    private void commit( RecordManager recordManager ) throws IOException
    {
        if ( commitNumber.incrementAndGet() % 2000 == 0 )
        {
            sync();
        }
    }


This is *BAD*. That means we may have some pending modifications of this
RDN index that are not written on disk when we stop the server or when
we have a crash :/

Adding a rdnIdx.sync() might be just enough to eliminate the corruptions
we see...


In any case, we still need the repair tool.


Mime
View raw message