hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5732) SFTP FileSystem
Date Wed, 06 May 2009 13:08:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12706412#action_12706412
] 

Steve Loughran commented on HADOOP-5732:
----------------------------------------

Basic Junit is fairly simple: extend TestCase and write a testXXX method for every test you
want. 

Functional testing of sftp will be tricker as it should only work if the test runner has been
told to do so, 

Have a look at org.apache.hadoop.fs.TestLocalFileSystem to see a filesystem test, and
org.apache.hadoop.fs.TestLocalFileSystemPermission to see one that uses System.getProperty()
to look for specific properties beginning with test. to control its operation -and which skips
any attempts to work on windows

for SFTP, a good test would be to take a filesystem URL as a system property (e.g. test.sftp.url)
and then have a series of tests that test out the various filesystem methods, starting with
the basic ones (Write that test first, get it working before doing the others)

testConnect: try to connect to that URL, close it
testCreateFile: connect, create a file, delete it ,close the filesystem
testFileExists: as above, check the file exists
testReadFile: connect, create a file, write to it, close the file, read it in, checksum the
data, delete it and close the FS
testRenameFile: create, rename a file
testStatFile: test getFileStatus

Although you can do all of this in one sequence, breaking it up is better for debugging problems
-when you see an error report, you can often guess what the problem is

And some trick ones 
testDeleteNonexistentFile  -delete a file that isn't there
testRenameNonexistentFile
testRenameOntoExistingFile
testConcurrentAccess: try to open the same file for writing twice, see what happens
...etc.

Thinking up tests that break the system are the most fun, and the way to reliable code.



> SFTP FileSystem
> ---------------
>
>                 Key: HADOOP-5732
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5732
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: fs
>         Environment: Any environment
>            Reporter: Íñigo Goiri
>            Priority: Minor
>         Attachments: HADOOP-FS-SFTP.patch, HADOOP-FS-SFTP.patch, ivy-for-hadoop-7532.patch,
ivy-for-hadoop-7532.patch, SFTPException.java, SFTPFileSystem.java, SFTPInputStream.java
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>
> I have implemented a FileSystem that supports SFTP. It uses JSch (http://www.jcraft.com/jsch/)
in order to manage SFTP.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message