hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-173) Recursively deleting a directory with millions of files makes NameNode unresponsive for other commands until the deletion completes
Date Tue, 18 Aug 2009 19:24:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744660#action_12744660
] 

Raghu Angadi commented on HDFS-173:
-----------------------------------


> First the target directory to be deleted will be removed from the directory tree, to
prevent further changes in that directory.

Just trying to see if there are some consistency issues :

What about other references to files under the tree. E.g. if a file is still being written,
there are references to it from lease manager, that could be an inconsistent view of the namesystem.
I think this one can be handled

What about log entry? Is the deletion log written at the beginning or the when the whole deletion
is completed? (At the beginning seems more accurate)..  Currently can recursive deletes fail
for any reason in the middle (and are those rolled back?)?

What is is the suspect for taking so long? Is it mainly the memory allocations?

> Recursively deleting a directory with millions of files makes NameNode unresponsive for
other commands until the deletion completes
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-173
>                 URL: https://issues.apache.org/jira/browse/HDFS-173
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>
> Delete a directory with millions of files. This could take several minutes (observed
12 mins for 9 million files). While the operation is in progress FSNamesystem lock is held
and the requests from clients are not handled until deletion completes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message