hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1143) Implement Background deletion
Date Sat, 15 May 2010 09:49:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12867845#action_12867845
] 

Hadoop QA commented on HDFS-1143:
---------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12444567/HDFS-1143.txt
  against trunk revision 944566.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    +1 core tests.  The patch passed core unit tests.

    -1 contrib tests.  The patch failed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/364/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/364/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/364/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/364/console

This message is automatically generated.

> Implement Background deletion
> -----------------------------
>
>                 Key: HDFS-1143
>                 URL: https://issues.apache.org/jira/browse/HDFS-1143
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.22.0
>            Reporter: Dmytro Molkov
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>
>         Attachments: HDFS-1143.txt
>
>
> Right now if you try to delete massive number of files from the namenode it will freeze
(sometimes for minutes). Most of the time is spent going through the blocks map and invalidating
all the blocks.
> This can probably be improved by having a background GC process. The deletion will basically
just remove the inode being deleted and then give the subtree that was just deleted to the
background thread running cleanup.
> This way the namenode becomes available for the clients soon after deletion, and all
the heavy operations are done in the background.
> Thoughts?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message