Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <29251769.68181273968224573.JavaMail.jira@thor>
Date: Sat, 15 May 2010 20:03:44 -0400 (EDT)
From: "Koji Noguchi (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-1143) Implement Background deletion
In-Reply-To: <18585901.9311273604022647.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12867935#action_12867935 ] 

Koji Noguchi commented on HDFS-1143:
------------------------------------

bq. Hey Koji,
bq. HDFS-173 help other clients because it releases the lock when removing blocks from time to time.

Ah.  "it will freeze (sometimes for minutes)." was meant for the client side and not the namenode?  I see.  Thanks.


> Implement Background deletion
> -----------------------------
>
>                 Key: HDFS-1143
>                 URL: https://issues.apache.org/jira/browse/HDFS-1143
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.22.0
>            Reporter: Dmytro Molkov
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>
>         Attachments: HDFS-1143.txt
>
>
> Right now if you try to delete massive number of files from the namenode it will freeze (sometimes for minutes). Most of the time is spent going through the blocks map and invalidating all the blocks.
> This can probably be improved by having a background GC process. The deletion will basically just remove the inode being deleted and then give the subtree that was just deleted to the background thread running cleanup.
> This way the namenode becomes available for the clients soon after deletion, and all the heavy operations are done in the background.
> Thoughts?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.