hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Date Tue, 01 Apr 2008 00:10:24 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583947#action_12583947

Konstantin Shvachko commented on HADOOP-3083:

# FSEditLog. Redundant:
import java.util.List;
import java.util.Iterator;
# FSDirectory.addFile() should return INodeFileUnderConstruction instead of INode.
# INode.getAbsoluteName() is deprecated and should not be used.
The full path for leases is necessary only when writing the INodeUnderConstruction to fsimage.

Otherwise, it is used in the current code and in the patch only for logging.
During saveImage() the full path of an INode is known so it should be remembered
next to the Lease and used in writeINodeUnderConstruction().
# I just noticed that we store each INodeFileUnderConstruction in fsimage twice, first as
a regular inode in the 
directory tree, and then in the file-under-construction section with additional file-under-construction
specific fields.
This is probably for a different issue.
# We also talked about how to efficiently implement the new file-under-construction 
collection that replaces current TreeSet of leases. Dhruba has a solution for that.
# File delete() is very inefficient because it uses removeLeases(), which traverses the whole

tree underneath current INode and collects all nodes in one array in order then to sort out
those that are being constructed.
In fact the traversal happens two times in collectSubtreeBlocks() and then in removeLeases().
The subtree should be traversed only once. We can introduce a method say
  collectSubtreeBlocksAndLeases(List<Block> v, List<INodeFileUnderConstruction>
which would collect both blocks and files under construction at the same time.

> Remove lease when file is renamed
> ---------------------------------
>                 Key: HADOOP-3083
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3083
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>            Priority: Blocker
>             Fix For: 0.17.0
>         Attachments: danglingLease.patch, leaseIgnoreExceptions.patch, leaseInode.patch
> When a file/directory is removed, the namenode should remove lease(s) for files that
were affected by the rename.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message