Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1469166748.1241644530406.JavaMail.jira@brutus>
Date: Wed, 6 May 2009 14:15:30 -0700 (PDT)
From: "Chris Douglas (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-5732) SFTP FileSystem
In-Reply-To: <410286140.1240503391005.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HADOOP-5732?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D127=
06591#action_12706591 ]=20

Chris Douglas commented on HADOOP-5732:
---------------------------------------

bq. It was based on the method used in the FTP FileSystem which used a runt=
ime exception. Should I use IOException then? For example if it throws an I=
OException at "exists" method, the API should be changed, I think that's wh=
y they used FTPException. In this case, should I just solve the exception l=
ocally? or throwing the SFTPException?

Yikes; I didn't know it did that. Yes, please throw IOException and not Run=
timeException. Applications won't expect the latter.

bq. Done, it extends FSInputStream. Should I modify something in order to d=
o it package-private?

Top-level classes without a visibility modifier are package-private, i.e. a=
re not visible to classes outside the package. No other classes should be c=
reating SFTPInputStream instances, right?

bq. I work with hadoop 0.19, which defines this methods, I prefer specifyin=
g it as deprecated.

I see. As a new feature, the earliest branch to which it should be committe=
d is 0.21, so the {{@Override}} annotations should fail. If you'd like to a=
ttach patches for 0.19 and/or 0.20 on this issue for others to use in their=
 own deployments, that's great, but the version committed to the mainline s=
hould not include these as they should have no callers.

bq. > Any particular reason for not supporting setting the working director=
y?
bq. It is the same problem with FTPFileSystem: "Directory on the server is =
changed to the parent directory of the file. The FTP client connection is c=
losed when close() is called on the FSDataInputStream."

That makes sense. Since the working directory can be a Path member in the F=
ileSystem handle, you could make all paths relative to that path instead of=
 the default directory, right? Applications referencing generic filesystems=
 might use the working directory abstraction, which would have unexpected r=
esults with this FileSystem. If FTPFileSystem has the same quirk and you've=
 documented it, it's OK if you want to leave it that way, but I'd lean towa=
rds supporting it. Applications that deal with generic FileSystems- like Di=
stCp- usually end up creating absolute paths to avoid edge cases like this =
one, but it would be nice if that were unnecessary. It's up to you.

bq. How do you plan to test this, incidentally? I'd recommend having some t=
est.* properties that, if set, would let you sftp back in to localhost or a=
nother named host.

The unit tests should not require the user running the tests to have an ssh=
 login on the host machine, neither should it be necessary to configure suc=
h a user. If jsch has a clever way to run its unit tests without actually p=
erforming these checks- perhaps in its development branch- that would be hu=
gely, hugely preferred. If that's not possible... then like KFS, there's no=
t much to be done to prevent regressions.

Please create a unified patch with the ivy changes that applies to 0.21. On=
ce that's done, you can submit the patch to automated testing using the "Su=
bmit Patch" link to the left.

> SFTP FileSystem
> ---------------
>
>                 Key: HADOOP-5732
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5732
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: fs
>         Environment: Any environment
>            Reporter: =C3=8D=C3=B1igo Goiri
>            Priority: Minor
>         Attachments: HADOOP-FS-SFTP.patch, HADOOP-FS-SFTP.patch, ivy-for-=
hadoop-7532.patch, ivy-for-hadoop-7532.patch, SFTPException.java, SFTPFileS=
ystem.java, SFTPInputStream.java
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>
> I have implemented a FileSystem that supports SFTP. It uses JSch (http://=
www.jcraft.com/jsch/) in order to manage SFTP.

--=20
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.