hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11306) Print remaining edit logs from buffer if edit log can't be rolled.
Date Tue, 10 Jan 2017 04:44:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813865#comment-15813865

Yongjun Zhang commented on HDFS-11306:

Hi [~jojochuang],

Thanks much for working on this issue!

Some comments of the patch:

# Suggest to print a summary WARN message at the beginning of {{dumpRemainingEditLogs()}},
stating something like "The edits buffer should have been flushed but there are still <numTxns>
unflushed. Below are the list of the unflushed transactions:".
# add a finally block and call
        IOUtils.cleanup(LOG, dis);
        IOUtils.cleanup(LOG, bis);
# Can we add couple of more different edits in the test?


> Print remaining edit logs from buffer if edit log can't be rolled.
> ------------------------------------------------------------------
>                 Key: HDFS-11306
>                 URL: https://issues.apache.org/jira/browse/HDFS-11306
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, namenode
>            Reporter: Wei-Chiu Chuang
>            Assignee: Wei-Chiu Chuang
>         Attachments: HDFS-11306.001.patch
> In HDFS-10943 [~yzhangal] reported that edit log can not be rolled due to unexpected
edit logs lingering in the buffer.
> Unable to root cause the bug, I propose that we dump the remaining edit logs in the buffer
into namenode log, before crashing namenode. Use this new capability to find the ops that
sneaks into the buffer unexpectedly, and hopefully catch the bug.
> This effort is orthogonal, but related to HDFS-11292, which adds additional informational
logs to help debug this issue.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message