hadoop-hdfs-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From t...@apache.org
Subject svn commit: r1125057 - in /hadoop/hdfs/trunk: CHANGES.txt src/docs/src/documentation/content/xdocs/hftp.xml src/docs/src/documentation/content/xdocs/site.xml
Date Thu, 19 May 2011 19:05:31 GMT
Author: todd
Date: Thu May 19 19:05:30 2011
New Revision: 1125057

URL: http://svn.apache.org/viewvc?rev=1125057&view=rev
Log:
HDFS-1957. Add documentation for HFTP. Contributed by Ari Rabkin.

Added:
    hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml
Modified:
    hadoop/hdfs/trunk/CHANGES.txt
    hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml

Modified: hadoop/hdfs/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/CHANGES.txt?rev=1125057&r1=1125056&r2=1125057&view=diff
==============================================================================
--- hadoop/hdfs/trunk/CHANGES.txt (original)
+++ hadoop/hdfs/trunk/CHANGES.txt Thu May 19 19:05:30 2011
@@ -848,6 +848,8 @@ Release 0.22.0 - Unreleased
 
     HDFS-1947. DFSClient should use mapreduce.task.attempt.id. (eli)
 
+    HDFS-1957. Add documentation for HFTP. (Ari Rabkin via todd)
+
   OPTIMIZATIONS
 
     HDFS-1140. Speedup INode.getPathComponents. (Dmytro Molkov via shv)

Added: hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml
URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml?rev=1125057&view=auto
==============================================================================
--- hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml (added)
+++ hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/hftp.xml Thu May 19 19:05:30
2011
@@ -0,0 +1,69 @@
+<?xml version="1.0"?>
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+
+<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" "http://forrest.apache.org/dtd/document-v20.dtd">
+
+
+<document>
+
+  <header>
+    <title> HFTP Guide</title>
+  </header>
+
+  <body>
+    <section>
+      <title> Introduction </title>
+      <p> HFTP is a Hadoop filesystem implementation that lets you read data from a
remote Hadoop HDFS cluster. 
+      The reads are done via HTTP, and data is sourced from DataNodes. 
+      HFTP is a read-only filesystem, and will throw exceptions if you try to use it to write
data or modify
+      the filesystem state.</p>     
+
+      <p>HFTP is primarily useful if you have multiple HDFS clusters with different
versions and you need to move data from one to another. HFTP is wire-compatible even between
different versions of HDFS. For example, you can do things like:
+      <code>hadoop distcp -i hftp://sourceFS:50070/src hdfs://destFS:50070/dest</code>.
Note that HFTP is read-only so the destination must be an HDFS filesystem. (Also, in this
example, the <code>distcp</code> should be run using the configuraton of the new
filesystem.)</p>
+      
+      <p>An extension, HSFTP, uses HTTPS by default. This means that data will be encrypted
in transit.</p>
+    </section>
+    
+    <section>
+      <title>Implementation</title>
+      <p>The code for HFTP lives in the Java class <code>org.apache.hadoop.hdfs.HftpFileSystem</code>.
Likewise, 
+      HSFTP is implemented in <code>org.apache.hadoop.hdfs.HsftpFileSystem</code>.
+      </p>
+    </section>
+    
+    <section>
+      <title> Configuration Options </title>
+      <table>
+        <tr>
+          <th>Name</th>
+          <th>Description</th>
+        </tr>
+        <tr>
+          <td>dfs.hftp.https.port</td>
+          <td>the HTTPS port on the remote cluster. If not set, HFTP will fall back
on
+          <code>dfs.https.port</code>.</td>
+        </tr>
+        <tr>
+          <td>hdfs.service.host_<strong>ip:port</strong></td>
+          <td>Specifies the service name (for the security subsystem) associated with
the HFTP filesystem
+          running at <strong>ip:port.</strong></td>
+        </tr>
+      </table>     
+    </section>
+  </body>
+</document>

Modified: hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml
URL: http://svn.apache.org/viewvc/hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml?rev=1125057&r1=1125056&r2=1125057&view=diff
==============================================================================
--- hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml (original)
+++ hadoop/hdfs/trunk/src/docs/src/documentation/content/xdocs/site.xml Thu May 19 19:05:30
2011
@@ -44,6 +44,7 @@ See http://forrest.apache.org/docs/linki
       <hdfs_imageviewer	label="Offline Image Viewer"	href="hdfs_imageviewer.html" />
       <hdfs_editsviewer	label="Offline Edits Viewer"	href="hdfs_editsviewer.html" />
       <hdfsproxy 			    label="HDFS Proxy" href="hdfsproxy.html"/>
+      <hftp 			    label="HFTP" href="hftp.html"/>
       <faultinject_framework label="Fault Injection"  href="faultinject_framework.html"
/> 
       <hdfs_libhdfs   		label="C API libhdfs" href="libhdfs.html" /> 
     </docs>



Mime
View raw message