hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2975) Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart).
Date Thu, 28 Aug 2014 05:16:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113347#comment-14113347

Uma Maheswara Rao G commented on HDFS-2975:

Feel free to file different bug for delete API on validating the issue there.

Original code what we fixed in delete is:
 deleteNow = collectedBlocks.size() <= BLOCK_DELETION_INCREMENT;
-      if (deleteNow) { // Perform small deletes right away
-        removeBlocks(collectedBlocks);
-      }
     } finally {
@@ -1990,9 +1985,7 @@
     try {
-      if (!deleteNow) {
-        removeBlocks(collectedBlocks); // Incremental deletion of blocks
-      }
+      removeBlocks(collectedBlocks); // Incremental deletion of blocks

But now, other APIs using removeBlocksAndUpdateSafeBlock on delete via rename with overwrite
or so.
Lets validate carefully whether this change really not needed for delete or it is a introduce
bug by other changes later.
Feel free to file separate Bug.

> Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash +
> -----------------------------------------------------------------------------------------------
>                 Key: HDFS-2975
>                 URL: https://issues.apache.org/jira/browse/HDFS-2975
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>            Reporter: Uma Maheswara Rao G
>            Assignee: Yi Liu
>         Attachments: HDFS-2975.001.patch
> When we rename the file with overwrite flag as true, it will delete the destination file
blocks. After deleting the blocks, whenever it releases the fsNameSystem lock, NN can give
the invalidation work to corresponding DNs to delete the blocks.
> Parallaly it will sync the rename related edits to editlog file. At this step before
NN sync the edits if NN crashes, NN can stuck into safemode on restart. This is because block
already deleted from the DN as part of invalidations. But dst file still exist as rename edits
not persisted in log file and no DN will report that blocks now.
> This is similar to HDFS-2815

This message was sent by Atlassian JIRA

View raw message