hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "HCFS/FileSystem-1.0-Tests" by JayVyas
Date Fri, 28 Jun 2013 19:20:46 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "HCFS/FileSystem-1.0-Tests" page has been changed by JayVyas:
https://wiki.apache.org/hadoop/HCFS/FileSystem-1.0-Tests?action=diff&rev1=8&rev2=9

  Initial Analysis of Hadoop FileSystem 1.0 Tests by the HFCS working group.
  
  ''The term "Impl Test" used below is intended to assert that the test validates a single
particular Hadoop FileSystem Implementation and should not belong with a Hadoop FS Test Library
that focuses on any FileSystem Plugins ability to implement the Hadoop FileSystem contract.''
+ ||''' Impl Test ''' ||'''Test''' ||'''What it does''' ||'''Runtime Test (main)''' ||
+ ||no FS ||TestDU.java ||Validates DU.java works properly || ||
+ ||no FS ||TestGlobPattern.java ||Validates Glob Pattern Matching Support || ||
+ || ||DFSCIOTest.java ||Why do we have this AND TestDFSIO ? ||x ||
+ ||no, but references org.apache.hadoop.hdfs.MiniDFSCluster ||TestDisableCache.java ||Validates
the ability to turn FS caching on and off || ||
+ ||no, but references org.apache.hadoop.hdfs.MiniDFSCluster ||TestHarFileSystem.java ||Validates
[[http://hadoop.apache.org/docs/stable/hadoop_archives.html|Hadoop Archives (HAR)]] Support
|| ||
+ || ||DistributedFSCheck.java ||Validates FS Consistency ||x ||
+ || ||FSMainOperationsBaseTest.java ||Validates FS Operations. Needs to be disambiguated
from FileSystemContractBaseTest || ||
+ ||no, but references  org.apache.hadoop.hdfs.MiniDFSCluster ||TestFileSystem.java ||Validates
Reads, Seeks and Writes via a M/R Job. Needs to be disambiguated from DFSCIOTest and TestDFSIO
||x ||
+ || ||FileSystemContractBaseTest.java ||Validates FS Impl adherence to FS Contract || ||
+ || ||FileSystemTestHelper.java ||Helper/Utility Class for FS Testing || ||
+ ||no, but references and configured with RawLocalFileSystem.||TestFsShellReturnCode.java
||Validates that Hadoop FS chmod, chown, chgrp commands work correctly || ||
+ ||no FS ||TestPath.java ||I think this is a Helper class for testing Paths. Seems to be
a mix of local and (perhaps) FS Tests || ||
+ || ||TestGetFileBlockLocations.java ||Validates the ability to determine Block/File Locations
|| ||
+ || ||TestTrash.java ||Validates FS support of Trash Operations || ||
+ ||no, but references org.apache.hadoop.hdfs.MiniDFSCluster ||TestCopyFiles.java ||Test Copy
Operations but needs to be disambiguated from FileSystemContractBaseTest and other FS Test
Classes || ||
+ || ||TestTruncatedInputBug.java ||Test for HADOOP-1489. Needs to be changed from Explicit
Local to FS || ||
+ || ||TestDFSIO.java ||Validate Read and Write Throughput ||x ||
+ ||no FS ||TestGlobPaths.java ||Tests Glob Paths. Needs to be disambiguated from other Glob
Tests || ||
+ ||no but depends on org.apache.hadoop.hdfs.MiniDFSCluster ||TestUrlStreamHandler.java ||High
level validation of FS Input Streams || ||
+ ||x ||AccumulatingReducer.java ||Reducer that accumulates values based on their type ||
||
+ ||x ||TestLocalDirAllocator.java ||Does not use FS. This is a LocalFileSystem test. || ||
+ ||x ||TestFileSystemCaching.java ||Does not use FS. This is a LocalFileSystem test. || ||
+ ||x ||TestLocalFileSystem.java ||Does not use FS. This is a LocalFileSystem test. || ||
+ ||x ||TestFileUtil.java ||Does not use FS. This is a LocalFileSystem test || ||
+ ||x ||TestLocalFileSystemPermission.java ||Does not use FS. This is a LocalFileSystem test
|| ||
+ ||x ||IOMapperBase.java ||Not sure where this is used. Seems redundant || ||
+ ||x ||TestChecksumFileSystem.java ||Does not use FS. This is a ChecksumFileSystem/InMemoryFileSystem
test || ||
+ ||x ||TestGlobExpander.java ||Helper class for Glob Operations. Not clear who uses it ||
||
+ ||x ||TestHardLink.java ||Does not use FS. We should remove. Validates Single and Multi
File Hard linking || ||
+ ||x ||TestFSMainOperationsLocalFileSystem.java ||Does Nothing. Recommend that we remove
this out of the FS Library || ||
  
+ 
+ 
- || ''' Impl Test ''' || '''Test''' || '''What it does''' || '''Runtime Test (main)''' ||
- ||  no FS  || TestDU.java || Validates DU.java works properly || ||
- || no FS  || TestGlobPattern.java || Validates Glob Pattern Matching Support || ||
- || || DFSCIOTest.java || Why do we have this AND TestDFSIO ? || x ||
- || no, but references org.apache.hadoop.hdfs.MiniDFSCluster || TestDisableCache.java ||
Validates the ability to turn FS caching on and off || ||
- || no, but references org.apache.hadoop.hdfs.MiniDFSCluster || TestHarFileSystem.java ||
Validates [[http://hadoop.apache.org/docs/stable/hadoop_archives.html | Hadoop Archives (HAR)]]
Support  || ||
- || || DistributedFSCheck.java || Validates FS Consistency|| x ||
- || || FSMainOperationsBaseTest.java || Validates FS Operations. Needs to be disambiguated
from FileSystemContractBaseTest ||  ||
- || no, but references  org.apache.hadoop.hdfs.MiniDFSCluster || TestFileSystem.java || Validates
Reads, Seeks and Writes via a M/R Job. Needs to be disambiguated from DFSCIOTest and TestDFSIO
|| x ||
- || || FileSystemContractBaseTest.java || Validates FS Impl adherence to FS Contract || ||
- || || FileSystemTestHelper.java || Helper/Utility Class for FS Testing || ||
- || || TestFsShellReturnCode.java || Validates that Hadoop FS chmod, chown, chgrp commands
work correctly || ||
- || no FS || TestPath.java || I think this is a Helper class for testing Paths. Seems to
be a mix of local and (perhaps) FS Tests || ||
- ||  || TestGetFileBlockLocations.java || Validates the ability to determine Block/File Locations
|| ||
- ||  || TestTrash.java || Validates FS support of Trash Operations || ||
- || no, but references org.apache.hadoop.hdfs.MiniDFSCluster || TestCopyFiles.java || Test
Copy Operations but needs to be disambiguated from FileSystemContractBaseTest and other FS
Test Classes|| ||
- ||  || TestTruncatedInputBug.java || Test for HADOOP-1489. Needs to be changed from Explicit
Local to FS || ||
- ||  || TestDFSIO.java || Validate Read and Write Throughput || x ||
- || no FS ||  TestGlobPaths.java || Tests Glob Paths. Needs to be disambiguated from other
Glob Tests || ||
- || no but depends on org.apache.hadoop.hdfs.MiniDFSCluster || TestUrlStreamHandler.java
|| High level validation of FS Input Streams || ||
- ||x|| AccumulatingReducer.java ||  Reducer that accumulates values based on their type ||
||
- ||x|| TestLocalDirAllocator.java || Does not use FS. This is a LocalFileSystem test. ||
||
- ||x|| TestFileSystemCaching.java || Does not use FS. This is a LocalFileSystem test. ||
||
- ||x|| TestLocalFileSystem.java || Does not use FS. This is a LocalFileSystem test. || ||
- ||x|| TestFileUtil.java || Does not use FS. This is a LocalFileSystem test || ||
- ||x|| TestLocalFileSystemPermission.java || Does not use FS. This is a LocalFileSystem test
|| ||
- ||x|| IOMapperBase.java || Not sure where this is used. Seems redundant || ||
- ||x|| TestChecksumFileSystem.java || Does not use FS. This is a ChecksumFileSystem/InMemoryFileSystem
test || ||
- ||x|| TestGlobExpander.java || Helper class for Glob Operations. Not clear who uses it ||
||
- ||x|| TestHardLink.java || Does not use FS. We should remove. Validates Single and Multi
File Hard linking || ||
- ||x|| TestFSMainOperationsLocalFileSystem.java || Does Nothing. Recommend that we remove
this out of the FS Library || ||
  
  Hadoop FS Test sub-packages
+ ||''' Impl Test ''' ||'''Test''' ||'''What it does''' ||
+ || ||permission ||Validates the Hadoop FS Plugin adherence to Permissions ||
+ || ||slive ||[[https://issues.apache.org/jira/browse/HDFS-708|Slive is a stress test for
the Hadoop FS]] ||
+ || ||loadGenerator ||A load testing tool for testing Hadoop FS behavior under different
mixes of read, write and list requests ||
+ ||x ||ftp ||Validates the Hadoop FS Plugin for an FTP Server ||
+ ||x ||kfs ||Validates the Hadoop FS Plugin for the Kosmos FS. Now defunct due to the [[https://wiki.apache.org/hadoop/HCFS|Quantcast
FS Plugin]] ||
+ ||x ||s3 ||Validates the Hadoop FS Plugin for the Amazon S3 Object Store ||
+ ||x ||s3native ||Pseudo S3 Implementation to allow testing of the Hadoop FS Plugin for S3
without connecting to Amazon S3 ||
  
- || ''' Impl Test ''' || '''Test''' || '''What it does''' ||
- || || permission || Validates the Hadoop FS Plugin adherence to Permissions ||
- || || slive || [[https://issues.apache.org/jira/browse/HDFS-708 | Slive is a stress test
for the Hadoop FS ]] ||
- || || loadGenerator || A load testing tool for testing Hadoop FS behavior under different
mixes of read, write and list requests ||
- || x || ftp || Validates the Hadoop FS Plugin for an FTP Server ||
- || x || kfs || Validates the Hadoop FS Plugin for the Kosmos FS. Now defunct due to the
[[https://wiki.apache.org/hadoop/HCFS | Quantcast FS Plugin]] ||
- || x || s3 || Validates the Hadoop FS Plugin for the Amazon S3 Object Store ||
- || x || s3native || Pseudo S3 Implementation to allow testing of the Hadoop FS Plugin for
S3 without connecting to Amazon S3 ||
  
+ 

Mime
View raw message