hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13074) TestS3AContractRootDir#testListEmptyRootDirectory fails with java.io.IOException: Root directory operation rejected
Date Fri, 29 Apr 2016 18:30:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264491#comment-15264491

Chris Nauroth commented on HADOOP-13074:

Hello [~Swagat].  These tests are intended to be run using an empty bucket, dedicated strictly
for the purpose of running these tests.  Quoting the same documentation:

The contents of each bucket will be destroyed during the test process: do not use the bucket
for any purpose other than testing. Furthermore, for s3a, all in-progress multi-part uploads
to the bucket will be aborted at the start of a test (by forcing fs.s3a.multipart.purge=true)
to clean up the temporary state of previously failed tests.

It is by design that the test aborts if it actually finds anything in the bucket that would
be deleted.  That protects people from accidentally pointing the test suite at a bucket containing
critical production data and deleting all of it.  Note the comment in {{AbstractContractRootDirectoryTest#testListEmptyRootDirectory}}:

  public void testListEmptyRootDirectory() throws IOException {
    //extra sanity checks here to avoid support calls about complete loss of data
    FileSystem fs = getFileSystem();
    Path root = new Path("/");
    FileStatus[] statuses = fs.listStatus(root);
    for (FileStatus status : statuses) {
      ContractTestUtils.assertDeleted(fs, status.getPath(), true);
    assertEquals("listStatus on empty root-directory returned a non-empty list",
        0, fs.listStatus(root).length);

I have verified that the test passes as expected when running with an empty bucket.

Does this make sense?  If so, I would like to resolve this issue as "Not a Problem".

> TestS3AContractRootDir#testListEmptyRootDirectory fails with java.io.IOException: Root
directory operation rejected
> -------------------------------------------------------------------------------------------------------------------
>                 Key: HADOOP-13074
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13074
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3, test
>    Affects Versions: 3.0.0
>            Reporter: Swagat Behera
> TestS3AContractRootDir#testListEmptyRootDirectory fails with java.io.IOException: Root
directory operation rejected . 
> Following https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md,
I had inputted a non empty S3 bucket with appropriate keys. 
> This test gets the status under the root and tries to delete all the folders under that
iteratively along with the root folder.
> When it tries to delete the root folder, ContractTestUtils::rejectRootOperation() throws
"Root directory operation rejected" exception. This is bound to happen since the allowRootOperation
flag is not set.
> [~cnauroth]
> Please let me know your comments on this. It seems that this test will always fail until
we start using ContractTestUtils::assertDeleted(allowRootOperations=True) .
> + [~fabbri], [~mackrorysd] for FYI

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message