hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mostafa Elhemali (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9361) Strictly define the expected behavior of filesystem APIs and write tests to verify compliance
Date Tue, 09 Jul 2013 16:01:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703419#comment-13703419
] 

Mostafa Elhemali commented on HADOOP-9361:
------------------------------------------

Thanks Steve for starting this. Personally I'm really glad to see more abstract testing-against-the-contract
efforts for the file systems in Hadoop so it's great to see this. My comments upon first reading
of the code (I didn't read the specs yet so no comments on those):

# Personal aesthetic point: I'd have personally preferred if the contract was not in XML config,
but just in code; and that isSupported() just took an enum that provided the feature. Code
is much easier and simpler to read in this case, and there's no real need to "configure" a
contract for a file system. And already you modify the configuration based on code in e.g.
LocalFSContract, so it's just easier to read if it was all in code.
# Typo: SUPPORTS_CONTAT should be SUPPORTS_CONCAT
# In assertPathExists(), I think you meant to include the ls() content in the fail() message
rather than just call it.
# In testConcatOnSel() - missed failing if an error isn't thrown.
# It would be great to catch() more specific exceptions in the Concat tests - i.e. have the
contract specify what exception to expect as well as just an exception being thrown.

                
> Strictly define the expected behavior of filesystem APIs and write tests to verify compliance
> ---------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-9361
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9361
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs, test
>    Affects Versions: 3.0.0, 2.1.0-beta
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-9361-001.patch, HADOOP-9361-002.patch
>
>
> {{FileSystem}} and {{FileContract}} aren't tested rigorously enough -while HDFS gets
tested downstream, other filesystems, such as blobstore bindings, don't.
> The only tests that are common are those of {{FileSystemContractTestBase}}, which HADOOP-9258
shows is incomplete.
> I propose 
> # writing more tests which clarify expected behavior
> # testing operations in the interface being in their own JUnit4 test classes, instead
of one big test suite. 
> # Having each FS declare via a properties file what behaviors they offer, such as atomic-rename,
atomic-delete, umask, immediate-consistency -test methods can downgrade to skipped test cases
if a feature is missing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message