hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bo Shi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5732) SFTP FileSystem
Date Thu, 20 May 2010 04:53:57 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869478#action_12869478

Bo Shi commented on HADOOP-5732:

listStatus() appears to cause some trouble: 

  public static void main(String [] args) throws Exception { 
    Configuration conf = new Configuration(); 
    conf.addResource(new Path("file:///home/bshi/test.xml")); 
    FileSystem fs = FileSystem.get(new URI("sftp://user:pass@somehost"), conf); 

    String rpath = "/home/vmc/archive/new/NEW-20100505.dat"; 
    System.out.println(rpath + " exists? " + fs.exists(new Path(rpath))); 

    /* Enabling this will cause the program to hang. 
    for(FileStatus s: fs.listStatus(new Path(rpath))) { 

Enabling the program to hang due to the following thread no reclaiming. Quick inspection reveals
no obvious places where a connection isn't being closed. 

"Connect thread SOMEREMOTEHOST session" prio=10 tid=0x00002aaaf8370800 nid=0x5191 runnable
   java.lang.Thread.State: RUNNABLE 
at java.net.SocketInputStream.socketRead0(Native Method) 
at java.net.SocketInputStream.read(SocketInputStream.java:129) 
at com.jcraft.jsch.IO.getByte(IO.java:82) 
at com.jcraft.jsch.Session.read(Session.java:795) 
at com.jcraft.jsch.Session.run(Session.java:1160) 
at java.lang.Thread.run(Thread.java:619) 

   Locked ownable synchronizers: 
- None 

This behavior is produced against a 0.20 backport, I'll try it with the official patch and
against trunk tomorrow.

> SFTP FileSystem
> ---------------
>                 Key: HADOOP-5732
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5732
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 0.20.0
>         Environment: Any environment
>            Reporter: Íñigo Goiri
>            Assignee: Íñigo Goiri
>            Priority: Minor
>             Fix For: 0.21.0
>         Attachments: HADOOP-5732.patch, HADOOP-5732.patch, HADOOP-5732.patch, HADOOP-5732.patch,
HADOOP-5732.patch, ivy-for-hadoop-7532.patch, ivy-for-hadoop-7532.patch
>   Original Estimate: 0h
>  Remaining Estimate: 0h
> I have implemented a FileSystem that supports SFTP. It uses JSch (http://www.jcraft.com/jsch/)
in order to manage SFTP.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message