hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-729) fsck option to list only corrupted files
Date Mon, 26 Oct 2009 05:54:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12769902#action_12769902

dhruba borthakur commented on HDFS-729:

> Is this a regular fsck with less output? That might still be prohibitively long and expensive
for regular poll

Yes, at this point, I am visualizing it as a regular fsck with less output.  The problem with
making this a new Namenode RPC is that this RPC would have an upper limit on the number of
corrupted files that can be returned via one single invocation of the RPC. This kind-of- reduces
the elegance of such an API. The alternative is to make this new RPC retrieve a max number
of corrupted files together with a cookie that can be used in the next invocation of the RPC
to retrieve the remaining set of corrupted files (similar to readdir).

If we use a regular fsck, it does not lock the NN for an extended period of time, neither
does it have a problem if the number of files to be retrieved is huge.

> fsck option to list only corrupted files
> ----------------------------------------
>                 Key: HDFS-729
>                 URL: https://issues.apache.org/jira/browse/HDFS-729
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
> An option to fsck to list only corrupted files will be very helpful for frequent monitoring.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message