hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bo Shi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5732) SFTP FileSystem
Date Thu, 20 May 2010 04:53:57 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869478#action_12869478
] 

Bo Shi commented on HADOOP-5732:
--------------------------------

listStatus() appears to cause some trouble: 



{code:title=Bar.java|borderStyle=solid} 
  public static void main(String [] args) throws Exception { 
    Configuration conf = new Configuration(); 
    conf.addResource(new Path("file:///home/bshi/test.xml")); 
    FileSystem fs = FileSystem.get(new URI("sftp://user:pass@somehost"), conf); 

    String rpath = "/home/vmc/archive/new/NEW-20100505.dat"; 
     
    System.out.println(rpath + " exists? " + fs.exists(new Path(rpath))); 

    /* Enabling this will cause the program to hang. 
    for(FileStatus s: fs.listStatus(new Path(rpath))) { 
      System.out.println(s); 
    } 
    */ 
    fs.close(); 
  } 
{code} 

Enabling the program to hang due to the following thread no reclaiming. Quick inspection reveals
no obvious places where a connection isn't being closed. 


{noformat} 
"Connect thread SOMEREMOTEHOST session" prio=10 tid=0x00002aaaf8370800 nid=0x5191 runnable
[0x000000004046e000..0x000000004046eb90] 
   java.lang.Thread.State: RUNNABLE 
at java.net.SocketInputStream.socketRead0(Native Method) 
at java.net.SocketInputStream.read(SocketInputStream.java:129) 
at com.jcraft.jsch.IO.getByte(IO.java:82) 
at com.jcraft.jsch.Session.read(Session.java:795) 
at com.jcraft.jsch.Session.run(Session.java:1160) 
at java.lang.Thread.run(Thread.java:619) 

   Locked ownable synchronizers: 
- None 
{noformat} 


This behavior is produced against a 0.20 backport, I'll try it with the official patch and
against trunk tomorrow.

> SFTP FileSystem
> ---------------
>
>                 Key: HADOOP-5732
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5732
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 0.20.0
>         Environment: Any environment
>            Reporter: Íñigo Goiri
>            Assignee: Íñigo Goiri
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-5732.patch, HADOOP-5732.patch, HADOOP-5732.patch, HADOOP-5732.patch,
HADOOP-5732.patch, ivy-for-hadoop-7532.patch, ivy-for-hadoop-7532.patch
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>
> I have implemented a FileSystem that supports SFTP. It uses JSch (http://www.jcraft.com/jsch/)
in order to manage SFTP.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message