hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HDFS-14513) FSImage which is saving should be clean while NameNode shutdown
Date Fri, 07 Jun 2019 16:46:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16858804#comment-16858804

Íñigo Goiri commented on HDFS-14513:

[^HDFS-14513.007.patch] LGTM.

> FSImage which is saving should be clean while NameNode shutdown
> ---------------------------------------------------------------
>                 Key: HDFS-14513
>                 URL: https://issues.apache.org/jira/browse/HDFS-14513
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: He Xiaoqiao
>            Assignee: He Xiaoqiao
>            Priority: Major
>         Attachments: HDFS-14513.001.patch, HDFS-14513.002.patch, HDFS-14513.003.patch,
HDFS-14513.004.patch, HDFS-14513.005.patch, HDFS-14513.006.patch, HDFS-14513.007.patch
> Checkpointer/FSImageSaver is regular tasks and dump NameNode meta to disk, at most per
hour by default. If it receive some command (e.g. transition to active in HA mode) it will
cancel checkpoint and delete tmp files using {{FSImage#deleteCancelledCheckpoint}}. However
if NameNode shutdown when checkpoint, the tmp files will not be cleaned anymore. 
> Consider there are 500m inodes+blocks, it could cost 5~10min to finish once checkpoint,
if we shutdown NameNode during checkpointing, fsimage checkpoint file will never be cleaned,
after long time, there could be many useless checkpoint files. So I propose that we should
add hook to clean that when shutdown.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message