hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From iwasak...@apache.org
Subject hadoop git commit: HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)
Date Thu, 03 Mar 2016 10:02:23 GMT
Repository: hadoop
Updated Branches:
  refs/heads/branch-2 9e87f59de -> 8095c612a

HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)

(cherry picked from commit 33a412e8a4ab729d588a9576fb7eb90239c6e383)

Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/8095c612
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/8095c612
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/8095c612

Branch: refs/heads/branch-2
Commit: 8095c612a39af51680df37c312f52f6afd43fc63
Parents: 9e87f59
Author: Masatake Iwasaki <iwasakims@apache.org>
Authored: Thu Mar 3 18:57:23 2016 +0900
Committer: Masatake Iwasaki <iwasakims@apache.org>
Committed: Thu Mar 3 18:57:41 2016 +0900

 hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt            |  3 +++
 .../hadoop-distcp/src/site/markdown/DistCp.md.vm       | 13 +++++++------
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
index 759bb13..ad9a1b7 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
@@ -1963,6 +1963,9 @@ Release 2.7.3 - UNRELEASED
     HDFS-8791. block ID-based DN storage layout can be very slow for datanode
     on ext4 (Chris Trezzo via kihwal)
+    HDFS-9048. DistCp documentation is out-of-dated
+    (Daisuke Kobayashi via iwasakims)

diff --git a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
index 31f0444..0faa975 100644
--- a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
+++ b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
@@ -412,12 +412,13 @@ $H3 Map sizing
 $H3 Copying Between Versions of HDFS
-  For copying between two different versions of Hadoop, one will usually use
-  HftpFileSystem. This is a read-only FileSystem, so DistCp must be run on the
-  destination cluster (more specifically, on NodeManagers that can write to the
-  destination cluster). Each source is specified as
-  `hftp://<dfs.http.address>/<path>` (the default `dfs.http.address` is
-  `<namenode>:50070`).
+  For copying between two different major versions of Hadoop (e.g. between 1.X
+  and 2.X), one will usually use WebHdfsFileSystem. Unlike the previous
+  HftpFileSystem, as webhdfs is available for both read and write operations,
+  DistCp can be run on both source and destination cluster.
+  Remote cluster is specified as `webhdfs://<namenode_hostname>:<http_port>`.
+  When copying between same major versions of Hadoop cluster (e.g. between 2.X
+  and 2.X), use hdfs protocol for better performance.
 $H3 MapReduce and other side-effects

View raw message