hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ankur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3246) FTP client over HDFS
Date Tue, 13 May 2008 08:40:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596323#action_12596323

Ankur commented on HADOOP-3246:

Trying to connect to a randomly generated port might result in getting connected to a a different
service running on the random port causing confusion in the commons FTPClient code and in
turn FTPFileSystem code.

As a simple fix I added the following line after bind() call in the MinaListener.start() method.


This sets the port correctly to the actual port the FTP server ended up listening on. As a
result we could do simple thing in our Test case like

MinaListener listener = (MinaListener) server.getServerContext().getListener("default");
int serverPort = listener.getPort();
 ftpFs = FileSystem.get(URI.create("ftp://admin:admin@localhost:" + serverPort), conf);

This works and I have tested it on my local machine.
I have created a JIRA issue for Mina -  https://issues.apache.org/jira/browse/FTPSERVER-134.

If acceptable then I can provide the updated ftpserver-core.jar with this  fix till the time
it gets pushed into their code line.

> FTP client over HDFS
> --------------------
>                 Key: HADOOP-3246
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3246
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: util
>    Affects Versions: 0.16.3
>            Reporter: Ankur
>            Priority: Minor
>         Attachments: commons-net-1.4.1.jar, ftpFileSystem_2.patch, ftplet-api-1.0.0-SNAPSHOT.jar,
ftpserver-core-1.0.0-SNAPSHOT.jar, ftpserver-server-1.0.0-SNAPSHOT.jar, mina-core-2.0.0-M2-20080407.124109-12.jar,
oro-2.0.8.jar, slf4j-api-1.4.3.jar, slf4j-log4j12-1.4.3.jar
> An FTP client that stores content directly into HDFS allows data from FTP serves to be
stored directly into HDFS instead of first copying the data locally and then uploading it
into HDFS. The benefits are apparent from an administrative perspective as large datasets
can be pulled from FTP servers with minimal human intervention.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message