hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1111) getCorruptFiles() should give some hint that the list is not complete
Date Tue, 29 Jun 2010 16:29:51 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883602#action_12883602

Sanjay Radia commented on HDFS-1111:

>I really think the correct design choice is to export basic APIs like getCorruptFiles()
as RPCs.
I suspect you have a misunderstanding of how the client side connects via RPC. 
We have no plans to expose the RPCs directly for now.
In order to allow tools to access such functionality it is not necessary to use the RPC directly;
Hdfs and DistributedFileSystem (which extend AbstractFileSystem and FileSystem)  are effectively
the client side library to access a NN. 
>On the other hand, if we do take getCorruptFiles() out of ClientProtocol, we will make
HDFS-1171 overly complicated or expensive.
Not if you add the method to Hdfs and DistributedFileSystem.
You simply need to make the case for adding getCorruptFIles to these two classes. It appears
that this functionality got slipped in as part of HDFS-1171.

> getCorruptFiles() should give some hint that the list is not complete
> ---------------------------------------------------------------------
>                 Key: HDFS-1111
>                 URL: https://issues.apache.org/jira/browse/HDFS-1111
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Rodrigo Schmidt
>            Assignee: Rodrigo Schmidt
>         Attachments: HADFS-1111.0.patch
> If the list of corruptfiles returned by the namenode doesn't say anything if the number
of corrupted files is larger than the call output limit (which means the list is not complete).
There should be a way to hint incompleteness to clients.
> A simple hack would be to add an extra entry to the array returned with the value null.
Clients could interpret this as a sign that there are other corrupt files in the system.
> We should also do some rephrasing of the fsck output to make it more confident when the
list is not complete and less confident when the list is known to be incomplete.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message