hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From iwasak...@apache.org
Subject hadoop git commit: HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)
Date Thu, 03 Mar 2016 10:01:55 GMT
Repository: hadoop
Updated Branches:
  refs/heads/trunk eb864d35d -> 33a412e8a


HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/33a412e8
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/33a412e8
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/33a412e8

Branch: refs/heads/trunk
Commit: 33a412e8a4ab729d588a9576fb7eb90239c6e383
Parents: eb864d3
Author: Masatake Iwasaki <iwasakims@apache.org>
Authored: Thu Mar 3 18:57:23 2016 +0900
Committer: Masatake Iwasaki <iwasakims@apache.org>
Committed: Thu Mar 3 18:57:23 2016 +0900

----------------------------------------------------------------------
 hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt            |  3 +++
 .../hadoop-distcp/src/site/markdown/DistCp.md.vm       | 13 +++++++------
 2 files changed, 10 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/33a412e8/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
index d922cd7..0ddecaf 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
@@ -2916,6 +2916,9 @@ Release 2.7.3 - UNRELEASED
     HDFS-8791. block ID-based DN storage layout can be very slow for datanode
     on ext4 (Chris Trezzo via kihwal)
 
+    HDFS-9048. DistCp documentation is out-of-dated
+    (Daisuke Kobayashi via iwasakims)
+
   OPTIMIZATIONS
 
   BUG FIXES

http://git-wip-us.apache.org/repos/asf/hadoop/blob/33a412e8/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
----------------------------------------------------------------------
diff --git a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
index 8f64ea2..bac5ecc 100644
--- a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
+++ b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
@@ -412,12 +412,13 @@ $H3 Map sizing
 
 $H3 Copying Between Versions of HDFS
 
-  For copying between two different versions of Hadoop, one will usually use
-  HftpFileSystem. This is a read-only FileSystem, so DistCp must be run on the
-  destination cluster (more specifically, on NodeManagers that can write to the
-  destination cluster). Each source is specified as
-  `hftp://<dfs.http.address>/<path>` (the default `dfs.http.address` is
-  `<namenode>:50070`).
+  For copying between two different major versions of Hadoop (e.g. between 1.X
+  and 2.X), one will usually use WebHdfsFileSystem. Unlike the previous
+  HftpFileSystem, as webhdfs is available for both read and write operations,
+  DistCp can be run on both source and destination cluster.
+  Remote cluster is specified as `webhdfs://<namenode_hostname>:<http_port>`.
+  When copying between same major versions of Hadoop cluster (e.g. between 2.X
+  and 2.X), use hdfs protocol for better performance.
 
 $H3 MapReduce and other side-effects
 


Mime
View raw message