hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Oriani (JIRA) <j...@apache.org>
Subject [jira] Commented: (HDFS-1032) Extend DFSck with an option to list corrupt files using API from HDFS-729
Date Wed, 17 Mar 2010 00:25:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12846231#action_12846231

André Oriani commented on HDFS-1032:

Sorry, I did not read your comment carefully  and because of the getListing call, i thought
that only directories could be supplied for path. In such case I need to adjust my strategy.

* Only files can be in corrupt state. 
* User cannot create files and directories  with the same name 

HDFS is complaint with both.

So I could write the follow method to tell me when the path is a corrupt file or it is a directory
which contains  a corrupted file .

public static final String FILE_SEPARATOR = "/" 

private boolean shallReturnCorruptFile(String path, String corruptFile)
  if (corruptFile.equals(path)) {
    //Exact match => path is a file
    return true;
  } else {
      //Ensure path ends with a file separator
      if (!path.endsWith(FILE_SEPARATOR) {
      if (corruptFile.startsWith(path){
        // It is a prefix => path was a directory
        return true;
  return false;

Of course, this code can be optimized if we realize that  String's methods  equals and startsWith
have the same behavior if compared strings have the same length.

> Extend DFSck with an option to list corrupt files using API from HDFS-729
> -------------------------------------------------------------------------
>                 Key: HDFS-1032
>                 URL: https://issues.apache.org/jira/browse/HDFS-1032
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: tools
>            Reporter: Rodrigo Schmidt
>            Assignee: André Oriani
>         Attachments: hdfs-1032_aoriani.patch
> HDFS-729 created a new API to namenode that returns the list of corrupt files.
> We can now extend fsck (DFSck.java) to add an option (e.g. --list_corrupt) that queries
the namenode using the new API and lists the corrupt blocks to the users.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message