hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3246) FTP client over HDFS
Date Tue, 29 Apr 2008 22:29:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12593148#action_12593148

Chris Douglas commented on HADOOP-3246:

A few points:
* +1 for this being a FileSystem
* Changing javac.version in build.xml from 1.5 to 1.6 should not be part of this patch
* ftp.server.host, ftp.server.username, and ftp.server.password should be {{o.a.h.conf.Configuration}}
properties, not system properties. Conditionally running the unit tests based on a switch
in build.xml is also not consistent with the existing unit tests; all the changes to build.xml
should probably be reverted.
* ftpClient.sh doesn't seem necessary; it certainly won't be if this is a FileSystem
* It looks like some testing code accidentally made it into the patch, in FtpShell:
+    conf.set("fs.default.name", "hdfs://agoel-pc:9000");
+    conf.set("fs.hdfs.impl", "org.apache.hadoop.dfs.DistributedFileSystem");
+    conf.set("hadoop.tmp.dir", "/tmp/hadoop-aankurgoel");
* Process trivia: CHANGES.txt is added by the committer when the patch goes in. The "Release
Notes" field in JIRA is filled out by the contributor
* FtpClient::EraserThread is a heavyweight way to hide the password. Unfortunately, java.io.Console::readPassword
is only in Java 1.6, and it disables echo via a (platform-dependent) native call, so it's
not clear how one would do this in Java 1.5. Leaving this as a configuration property is probably
sufficient until we can move to Java 1.6; as a FileSystem, it needs to be a Configuration
property, anyway.
* If you wanted to keep some of the code for a FTP client separate from existing tools, take
a look at {{o.a.h.util.ToolBase}} to pick up some of the generic option parsing common to
most hadoop utilities

> FTP client over HDFS
> --------------------
>                 Key: HADOOP-3246
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3246
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: util
>    Affects Versions: 0.16.3
>            Reporter: Ankur
>            Priority: Minor
>         Attachments: ftpClient.patch
> An FTP client that stores content directly into HDFS allows data from FTP serves to be
stored directly into HDFS instead of first copying the data locally and then uploading it
into HDFS. The benefits are apparent from an administrative perspective as large datasets
can be pulled from FTP servers with minimal human intervention.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message