hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13074) TestS3AContractRootDir#testListEmptyRootDirectory fails with java.io.IOException: Root directory operation rejected
Date Fri, 29 Apr 2016 18:30:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264491#comment-15264491
] 

Chris Nauroth commented on HADOOP-13074:
----------------------------------------

Hello [~Swagat].  These tests are intended to be run using an empty bucket, dedicated strictly
for the purpose of running these tests.  Quoting the same documentation:

{quote}
The contents of each bucket will be destroyed during the test process: do not use the bucket
for any purpose other than testing. Furthermore, for s3a, all in-progress multi-part uploads
to the bucket will be aborted at the start of a test (by forcing fs.s3a.multipart.purge=true)
to clean up the temporary state of previously failed tests.
{quote}

It is by design that the test aborts if it actually finds anything in the bucket that would
be deleted.  That protects people from accidentally pointing the test suite at a bucket containing
critical production data and deleting all of it.  Note the comment in {{AbstractContractRootDirectoryTest#testListEmptyRootDirectory}}:

{code}
  @Test
  public void testListEmptyRootDirectory() throws IOException {
    //extra sanity checks here to avoid support calls about complete loss of data
    skipIfUnsupported(TEST_ROOT_TESTS_ENABLED);
    FileSystem fs = getFileSystem();
    Path root = new Path("/");
    FileStatus[] statuses = fs.listStatus(root);
    for (FileStatus status : statuses) {
      ContractTestUtils.assertDeleted(fs, status.getPath(), true);
    }
    assertEquals("listStatus on empty root-directory returned a non-empty list",
        0, fs.listStatus(root).length);
  }
{code}

I have verified that the test passes as expected when running with an empty bucket.

Does this make sense?  If so, I would like to resolve this issue as "Not a Problem".

> TestS3AContractRootDir#testListEmptyRootDirectory fails with java.io.IOException: Root
directory operation rejected
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-13074
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13074
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3, test
>    Affects Versions: 3.0.0
>            Reporter: Swagat Behera
>
> TestS3AContractRootDir#testListEmptyRootDirectory fails with java.io.IOException: Root
directory operation rejected . 
> Following https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md,
I had inputted a non empty S3 bucket with appropriate keys. 
> This test gets the status under the root and tries to delete all the folders under that
iteratively along with the root folder.
> When it tries to delete the root folder, ContractTestUtils::rejectRootOperation() throws
"Root directory operation rejected" exception. This is bound to happen since the allowRootOperation
flag is not set.
> [~cnauroth]
> Please let me know your comments on this. It seems that this test will always fail until
we start using ContractTestUtils::assertDeleted(allowRootOperations=True) .
> + [~fabbri], [~mackrorysd] for FYI



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message