Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <803003878.1256536440573.JavaMail.jira@brutus>
Date: Mon, 26 Oct 2009 05:54:00 +0000 (UTC)
From: "dhruba borthakur (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-729) fsck option to list only corrupted
 files
In-Reply-To: <526750676.1256282639364.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12769902#action_12769902 ] 

dhruba borthakur commented on HDFS-729:
---------------------------------------

> Is this a regular fsck with less output? That might still be prohibitively long and expensive for regular poll

Yes, at this point, I am visualizing it as a regular fsck with less output.  The problem with making this a new Namenode RPC is that this RPC would have an upper limit on the number of corrupted files that can be returned via one single invocation of the RPC. This kind-of- reduces the elegance of such an API. The alternative is to make this new RPC retrieve a max number of corrupted files together with a cookie that can be used in the next invocation of the RPC to retrieve the remaining set of corrupted files (similar to readdir).

If we use a regular fsck, it does not lock the NN for an extended period of time, neither does it have a problem if the number of files to be retrieved is huge.


> fsck option to list only corrupted files
> ----------------------------------------
>
>                 Key: HDFS-729
>                 URL: https://issues.apache.org/jira/browse/HDFS-729
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> An option to fsck to list only corrupted files will be very helpful for frequent monitoring.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.