hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jay vyas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10461) Runtime HCFS injector for FileSystem tests
Date Fri, 04 Apr 2014 12:18:14 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

jay vyas updated HADOOP-10461:
------------------------------

    Description: 
Currently alot of manual inheritance and stub classes are required in order to run the FileSystemBaseContract
and FSMainOperations tests. 

For example:  This is a sampling of the classes necessary to glue Local FS implementations
into the Existing generic FileSystem test classes: 

{noformat}

.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFcLocalFsPermission.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFcLocalFsUtil.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFSMainOperationsLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystemPermission.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFsFCStatistics.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFSFileContextCreateMkdir.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFSFileContextMainOperations.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFSFileContext.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFSFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcCreateMkdirLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcMainOperationsLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcPermissionsLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFSMainOperationsLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemWithAuthorityLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsWithAuthorityLocalFs.java
{noformat}

Lets provide a Guice or other DI based injector for HCFS tests which

1) Injects the file system at runtime.
2) Can easily be adopted for other FileSystems.
3) Can read in System properties to skip certain tests , thus providing support for the type
of variability that we know FileSystem tests require.  These can then be used to implement
the semantics of HADOOP-9361.

Ideally, we could replace RawLocalFileSystem tests with this injector as a second follow up
patch to this, it would reduce the overall amount of code required, probably.










  was:
Currently alot of manual inheritance and stub classes are required in order to run the FileSystemBaseContract
and FSMainOperations tests. 

For example:  This is a sampling of the classes necessary to glue Local FS implementations
into the Existing generic FileSystem test classes: 

{noformat}

.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFcLocalFsPermission.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFcLocalFsUtil.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFSMainOperationsLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystemPermission.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFsFCStatistics.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFSFileContextCreateMkdir.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFSFileContextMainOperations.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFSFileContext.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFSFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcCreateMkdirLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcMainOperationsLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcPermissionsLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFSMainOperationsLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemWithAuthorityLocalFileSystem.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsLocalFs.java
.//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsWithAuthorityLocalFs.java
{noformat}

Lets provide a Guice or other DI based injector for HCFS tests which

1) Injects the file system at runtime.
2) Can easily be adopted for other FileSystems.
3) Can read in System properties to skip certain tests , thus providing support for the type
of variability that we know FileSystem tests require.

Ideally, we could replace RawLocalFileSystem tests with this injector as a second follow up
patch to this, it would reduce the overall amount of code required, probably.











> Runtime HCFS injector for FileSystem tests
> ------------------------------------------
>
>                 Key: HADOOP-10461
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10461
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: jay vyas
>            Priority: Minor
>
> Currently alot of manual inheritance and stub classes are required in order to run the
FileSystemBaseContract and FSMainOperations tests. 
> For example:  This is a sampling of the classes necessary to glue Local FS implementations
into the Existing generic FileSystem test classes: 
> {noformat}
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFcLocalFsPermission.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFcLocalFsUtil.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestFSMainOperationsLocalFileSystem.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystemPermission.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFsFCStatistics.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFSFileContextCreateMkdir.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFSFileContextMainOperations.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFS.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFSFileContext.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/TestSymlinkLocalFSFileSystem.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcCreateMkdirLocalFs.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcMainOperationsLocalFs.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcPermissionsLocalFs.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFSMainOperationsLocalFileSystem.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemLocalFileSystem.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemWithAuthorityLocalFileSystem.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsLocalFs.java
> .//hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsWithAuthorityLocalFs.java
> {noformat}
> Lets provide a Guice or other DI based injector for HCFS tests which
> 1) Injects the file system at runtime.
> 2) Can easily be adopted for other FileSystems.
> 3) Can read in System properties to skip certain tests , thus providing support for the
type of variability that we know FileSystem tests require.  These can then be used to implement
the semantics of HADOOP-9361.
> Ideally, we could replace RawLocalFileSystem tests with this injector as a second follow
up patch to this, it would reduce the overall amount of code required, probably.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message