hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5433) When reloading fsimage during checkpointing, we should clear existing snapshottable directories
Date Mon, 28 Oct 2013 03:12:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806530#comment-13806530

Vinay commented on HDFS-5433:

Thanks for filing this Jira Aaron.

Patch looks good to me.   

Small Nits:
Duplicate assertions in  TestCheckpointsWithSnapshots.testCheckpoint()
{code:java}+      assertEquals(1, nnSnapshotManager.getNumSnapshots());
+      assertEquals(1, nnSnapshotManager.getNumSnapshots());{code}
{code:java}+      assertEquals(0, nnSnapshotManager.getNumSnapshots());
+      assertEquals(0, nnSnapshotManager.getNumSnapshots());

+1 on addressing these nits.

> When reloading fsimage during checkpointing, we should clear existing snapshottable directories
> -----------------------------------------------------------------------------------------------
>                 Key: HDFS-5433
>                 URL: https://issues.apache.org/jira/browse/HDFS-5433
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: snapshots
>    Affects Versions: 2.2.0
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>            Priority: Critical
>         Attachments: HDFS-5433.patch
> The complete set of snapshottable directories are referenced both via the file system
tree and in the SnapshotManager class. It's possible that when the 2NN performs a checkpoint,
it will reload its in-memory state based on a new fsimage from the NN, but will not clear
the set of snapshottable directories referenced by the SnapshotManager. In this case, the
2NN will write out an fsimage that cannot be loaded, since the integer written to the fsimage
indicating the number of snapshottable directories will be out of sync with the actual number
of snapshottable directories serialized to the fsimage.
> This is basically the same as HDFS-3835, but for snapshottable directories instead of
delegation tokens.

This message was sent by Atlassian JIRA

View raw message