hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7917) Use file to replace data dirs in test to simulate a disk failure.
Date Mon, 23 Mar 2015 18:23:52 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lei (Eddy) Xu updated HDFS-7917:
--------------------------------
    Attachment: HDFS-7917.002.patch

Hi, [~cnauroth] 

Thanks a lot for your reviews and inputs. It is pity that this patch still cannot work on
Windows. But I think this patch can offer its own value, for example:

* {{git clean}} is per jenkins job. This patch can prevent a failure in {{@After}} causing
issues for the following unrelated tests.
* {{DataNodeTestUtils#injectDataDirFailure()}} is more verbose about the purpose. And we can
improve its functionality (i.e., supporting Windows) inside this function in the future. 

[~cnauroth], If you are OK with it, I would like to get this patch in. Would you mind give
another review? I updated the patch based on your comments. Thanks!



> Use file to replace data dirs in test to simulate a disk failure. 
> ------------------------------------------------------------------
>
>                 Key: HDFS-7917
>                 URL: https://issues.apache.org/jira/browse/HDFS-7917
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 2.6.0
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>            Priority: Minor
>         Attachments: HDFS-7917.000.patch, HDFS-7917.001.patch, HDFS-7917.002.patch
>
>
> Currently, in several tests, e.g., {{TestDataNodeVolumeFailureXXX}} and {{TestDataNotHowSwapVolumes}},
 we simulate a disk failure by setting a directory's executable permission as false. However,
it raises the risk that if the cleanup code could not be executed, the directory can not be
easily removed by Jenkins job. 
> Since in {{DiskChecker#checkDirAccess}}:
> {code}
> private static void checkDirAccess(File dir) throws DiskErrorException {
>     if (!dir.isDirectory()) {
>       throw new DiskErrorException("Not a directory: "
>                                    + dir.toString());
>     }
>     checkAccessByFileMethods(dir);
>   }
> {code}
> We can replace the DN data directory as a file to achieve the same fault injection goal,
while it is safer for cleaning up in any circumstance. Additionally, as [~cnauroth] suggested:

> bq. That might even let us enable some of these tests that are skipped on Windows, because
Windows allows access for the owner even after permissions have been stripped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message