hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Paduano (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-12696) Add Tests for S3FileSystem Contract
Date Fri, 08 Jan 2016 20:49:39 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-12696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthew Paduano updated HADOOP-12696:
-------------------------------------
    Attachment: log.hdfs
                log.s3n
                log.s3a

to copy.

   I dunno.   maybe.  maybe not.   I did see ContractOptions.IS_BLOBSTORE
in a
   few places in the test code when I was reviewing the failures.  But it
didn't seem
   like it would have fixed the test issues to me whichever way it pointed.

   But, it is easy enough to run the tests with various contracts.   I
removed
   all the forced skip @overrides from the test code and re-ran the tests
with
   the original s3.xml (== hdfs.xml) and also with s3a.xml and s3n.xml.
The
   same 10 tests still have issues, but the issues are different ;)

   I attached the output from all three test runs to this email.   summary:

   hdfs     Tests run: 47, Failures: 3, Errors: 7, Skipped: 0
   s3n      Tests run: 47, Failures: 2, Errors: 6, Skipped: 2
   s3a      Tests run: 47, Failures: 3, Errors: 7, Skipped: 2

   the s3n case does seem to account for two of the errors and
   instead they are "auto skipped".  testCreatedFileIsImmediatelyVisible
   (which is the atomic file option I think) and
testOverwriteNonEmptyDirectory
   (not sure of the option).   I am not sure I understand the skip code...
   it is sort of like a intermediate state between fail and pass...


matt


> Add Tests for S3FileSystem Contract
> -----------------------------------
>
>                 Key: HADOOP-12696
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12696
>             Project: Hadoop Common
>          Issue Type: Test
>          Components: tools
>    Affects Versions: 2.7.0
>            Reporter: Matthew Paduano
>            Assignee: Matthew Paduano
>              Labels: S3
>             Fix For: 2.8.0
>
>         Attachments: HADOOP-12696.01.patch, log.fail2, log.hdfs, log.s3a, log.s3n, log.succ
>
>
> The regression fixed by HADOOP-12689 had no unit tests to expose the problem.   Add filesystem
tests according to http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/testing.html
for the s3 scheme.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message