hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mingjian Deng <koven2...@gmail.com>
Subject data loss when splitLog()
Date Tue, 18 Oct 2011 14:40:51 GMT
Hi:
    There is a case cause data loss in our cluster. We block in splitLog
because some error in our hdfs and we kill master. Some hlog files were
moved from .logs to .oldlogs before them were wrote to .recovered.edits. So
rs couldn't replay these files.
    In HLogSplitter.java, we found:
    ...
    archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
    } finally {
      LOG.info("Finishing writing output logs and closing down.");
      splits = outputSink.finishWritingAndClose();
    }
    Why archiveLogs before outputSink.finishWritingAndClose()? Did these
hlog files mv to .oldlogs and couldn't be split next startup if write
threads failed but archiveLog success?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message