hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sathish (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5428) under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode
Date Thu, 07 Nov 2013 02:12:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815559#comment-13815559

sathish commented on HDFS-5428:

But I am little uncomfortable for managing leases for snapshotted files as they are readonly
files, no need of leases. If all others ok on that point, I will not object.

After this point ,Uma and me discussed the same points what Jing has mentioned in the HDFS-5428-000.patch.
It is better way to maintain the leases regarding the snapshot files in snapshot manager,As
the responsibility of lease manager is to maintain the leases for open files for write.with
the current implementation snapshots are read only,so there is no need to maintain the leases
for snapshotted files in lease manager.so it is better to maintain the leases regarding the
snapshotted files in snapshot manager.

+1 patch looks good
I will verify this patch in my env once.

> under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode
> ----------------------------------------------------------------------------------------
>                 Key: HDFS-5428
>                 URL: https://issues.apache.org/jira/browse/HDFS-5428
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Vinay
>            Assignee: Vinay
>         Attachments: HDFS-5428-v2.patch, HDFS-5428.000.patch, HDFS-5428.patch
> 1. allow snapshots under dir /foo
> 2. create a file /foo/test/bar and start writing to it
> 3. create a snapshot s1 under /foo after block is allocated and some data has been written
to it
> 4. Delete the directory /foo/test
> 5. wait till checkpoint or do saveNameSpace
> 6. restart NN.
> NN enters to safemode.
> Analysis:
> Snapshot nodes loaded from fsimage are always complete and all blocks will be in COMPLETE
> So when the Datanode reports RBW blocks those will not be updated in blocksmap.
> Some of the FINALIZED blocks will be marked as corrupt due to length mismatch.

This message was sent by Atlassian JIRA

View raw message