hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cosmin Lehene (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-2437) Refactor HLog splitLog
Date Tue, 13 Apr 2010 12:06:49 GMT
Refactor HLog splitLog

                 Key: HBASE-2437
                 URL: https://issues.apache.org/jira/browse/HBASE-2437
             Project: Hadoop HBase
          Issue Type: Bug
          Components: master
    Affects Versions: 0.21.0
            Reporter: Cosmin Lehene
            Assignee: Cosmin Lehene
             Fix For: 0.21.0

the HLog.splitLog got really long and complex and hard to verify for correctness. 
I started to refactor it and also ported changes from hbase-2337 that deals with premature
deletion of log files in case of errors. Further improvements will be possible, however the
scope of this issue is to clean the code and make it behave correctly (i.e. not lose any edits)

Added a suite of unit tests that might be ported to 0.20 as well.

Added a setting (hbase.skip.errors - feel free to suggest a better name) that, when set to
false will make the process less tolerant to failures or corrupted files:  in case a log file
is corrupted or an error stops the process from consistently splitting the log, will abort
the entire operation to avoid losing any edits. When hbase.skip.errors is on any corrupted
files will be partially parsed and then moved to the corrupted logs archive (see hbase-2337).

Like hbase-2337 the splitLog method will first split all the logs and then proceed to archive
them. If any splitted log file (oldlogfile.log) that is the result of an earlier splitLog
attempt is found in the region directory, it will be deleted - this is safe since we won't
move the original log files until the splitLog process completes.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message