hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Oriani (JIRA) <j...@apache.org>
Subject [jira] Commented: (HDFS-1032) Extend DFSck with an option to list corrupt files using API from HDFS-729
Date Tue, 16 Mar 2010 00:48:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845629#action_12845629

André Oriani commented on HDFS-1032:

First I would like to thank you both for the inputs.

I plan to replace  _corrupted_  by _corrupt_ in order to be consistent with the rest of the
code. I also plan to rename the method to listCorruptFiles() to distinguish it better from
the boolean field.

The mechanism to handle options is quite simple. From the method printUsage() of DFSck class
I understand the options -move, -delete and -openforwrite are also mutually exclusive. The
same can said of -locations and -racks. Should I also validate those cases?

I thought the same thing about how fsck() would call showCorruptedFiles()  at the first, but
I realized I needed to validate the path supplied by user before proceeding. So I reused the
call to namenode.getListing(path) for that.

Imagine user supplied  "/au". Using your suggestion and current implementation of  showCorruptedFiles()
 "/audio/music.ogg" could be returned, but that is not the user intended for.  WoW! I found
a bug! If both "/audio" and "/audiobook"  are valid directories, but user supplied "/audio"
, I reckon only files under the first one should be returned. That will no happen with my
code.   I wonder if I just ensure that the last character in the path is '/' will solve the

Regarding the code formatting infractions, I guess the will vanish if I set my Eclipse to
format using Java Conventions and 2 spaces for indentation. 

> Extend DFSck with an option to list corrupt files using API from HDFS-729
> -------------------------------------------------------------------------
>                 Key: HDFS-1032
>                 URL: https://issues.apache.org/jira/browse/HDFS-1032
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: tools
>            Reporter: Rodrigo Schmidt
>         Attachments: hdfs-1032_aoriani.patch
> HDFS-729 created a new API to namenode that returns the list of corrupt files.
> We can now extend fsck (DFSck.java) to add an option (e.g. --list_corrupt) that queries
the namenode using the new API and lists the corrupt blocks to the users.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message