hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cosmin Lehene (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-2958) When hbase.hlog.split.skip.errors is set to false, we fail the split but thats it
Date Thu, 08 Jan 2015 08:34:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268974#comment-14268974
] 

Cosmin Lehene commented on HBASE-2958:
--------------------------------------

[~stack] can you double check if this this still the case? 

Based on doc 
{noformat}
1.6.5.3.1. Handling of Errors During Log Splitting
If you set the hbase.hlog.split.skip.errors option to true, errors are treated as follows:
Any error encountered during splitting will be logged.
The problematic WAL log will be moved into the .corrupt directory under the hbase rootdir,
Processing of the WAL will continue
{noformat}

also 
in {{WALSplitter.getNextLogLine}} we throw {{CorruptedLogFileException}} if we're set to skip
errors and then in

in {{WALSplitter.splitLogFile}} we mark the log as corrupted accordingly
{code}
    } catch (CorruptedLogFileException e) {
      LOG.warn("Could not parse, corrupted log file " + logPath, e);
      csm.getSplitLogWorkerCoordination().markCorrupted(rootDir,
        logfile.getPath().getName(), fs);
      isCorrupted = true;
{code}


> When hbase.hlog.split.skip.errors is set to false, we fail the split but thats it
> ---------------------------------------------------------------------------------
>
>                 Key: HBASE-2958
>                 URL: https://issues.apache.org/jira/browse/HBASE-2958
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>
> When hbase.hlog.split.skip.errors is set to false, if we encounter an error splitting,
splitting stops and exception is let propagate up the stack.  I see that its caught in the
new MasterFileSystem class and logged, but thats it.  It would seem processing continues BUT
we've dropped the edits in the split.  We need to do better (default is hbase.hlog.split.skip.errors
set to false -- i.e. skip errors but keep going).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message