hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Xiaoqiao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14513) FSImage which is saving should be clean while NameNode shutdown
Date Wed, 29 May 2019 09:45:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850707#comment-16850707

He Xiaoqiao commented on HDFS-14513:

[~elgoiri] Thanks for taking a look.
I would like to offer detail information about this issue:
a. Checkpointer/FSImageSaver is regular tasks, at most per hour by default.
b. If meta of NameNode is huge, for example 500m inodes+blocks, it will take for a while to
persist to fsimage file.
c. If we shutdown NameNode during checkpointing, fsimage checkpiont file will be not cleaned.
After long time, there could be many useless checkpoint files.
[~elgoiri]  [^HDFS-14351.002.patch] use {{org.apache.hadoop.util.ShutdownHookManager}} to
manage the hook logic, would you take another look. Thanks.

> FSImage which is saving should be clean while NameNode shutdown
> ---------------------------------------------------------------
>                 Key: HDFS-14513
>                 URL: https://issues.apache.org/jira/browse/HDFS-14513
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: He Xiaoqiao
>            Assignee: He Xiaoqiao
>            Priority: Major
>         Attachments: HDFS-14351.002.patch, HDFS-14513.001.patch
> FSImage checkpoint file which is saving could not be clean while NameNode shutdown at
the same time.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message