hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7917) Use file to replace data dirs in test to simulate a disk failure.
Date Mon, 23 Mar 2015 18:23:52 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Lei (Eddy) Xu updated HDFS-7917:
    Attachment: HDFS-7917.002.patch

Hi, [~cnauroth] 

Thanks a lot for your reviews and inputs. It is pity that this patch still cannot work on
Windows. But I think this patch can offer its own value, for example:

* {{git clean}} is per jenkins job. This patch can prevent a failure in {{@After}} causing
issues for the following unrelated tests.
* {{DataNodeTestUtils#injectDataDirFailure()}} is more verbose about the purpose. And we can
improve its functionality (i.e., supporting Windows) inside this function in the future. 

[~cnauroth], If you are OK with it, I would like to get this patch in. Would you mind give
another review? I updated the patch based on your comments. Thanks!

> Use file to replace data dirs in test to simulate a disk failure. 
> ------------------------------------------------------------------
>                 Key: HDFS-7917
>                 URL: https://issues.apache.org/jira/browse/HDFS-7917
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 2.6.0
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>            Priority: Minor
>         Attachments: HDFS-7917.000.patch, HDFS-7917.001.patch, HDFS-7917.002.patch
> Currently, in several tests, e.g., {{TestDataNodeVolumeFailureXXX}} and {{TestDataNotHowSwapVolumes}},
 we simulate a disk failure by setting a directory's executable permission as false. However,
it raises the risk that if the cleanup code could not be executed, the directory can not be
easily removed by Jenkins job. 
> Since in {{DiskChecker#checkDirAccess}}:
> {code}
> private static void checkDirAccess(File dir) throws DiskErrorException {
>     if (!dir.isDirectory()) {
>       throw new DiskErrorException("Not a directory: "
>                                    + dir.toString());
>     }
>     checkAccessByFileMethods(dir);
>   }
> {code}
> We can replace the DN data directory as a file to achieve the same fault injection goal,
while it is safer for cleaning up in any circumstance. Additionally, as [~cnauroth] suggested:

> bq. That might even let us enable some of these tests that are skipped on Windows, because
Windows allows access for the owner even after permissions have been stripped.

This message was sent by Atlassian JIRA

View raw message