hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11366) Clean up old .ckpt files after saveNamespace
Date Tue, 24 Jan 2017 20:28:26 GMT
Xiao Chen created HDFS-11366:

             Summary: Clean up old .ckpt files after saveNamespace
                 Key: HDFS-11366
                 URL: https://issues.apache.org/jira/browse/HDFS-11366
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs, namenode
    Affects Versions: 2.6.0
            Reporter: Xiao Chen
            Assignee: Xiao Chen

Checkpoints are done in the NN by writing to {{fsimage.ckpt_TXID}} files, and rename to {{fsimage_TXID}}
files upon success.

If a checkpoint fails half way, the fsimage.ckpt_ file will be left on disk. There is no logic
to clean it up at all.

After talking with [~atm], I understand the historical reason for not immediately cleaning
up those files, since they maybe useful for disaster recovery.

But feels like cleaning those ckpt files after a successful checkpoint, with a larger TXID
threshold is also safe to do.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message