hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sj...@apache.org
Subject [46/50] [abbrv] hadoop git commit: HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula.
Date Tue, 22 Nov 2016 21:34:03 GMT
HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/afcf8d38
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/afcf8d38
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/afcf8d38

Branch: refs/heads/HADOOP-13070
Commit: afcf8d38e750f935c06629e641a1321b79c4cace
Parents: a926f89
Author: Brahma Reddy Battula <brahma@apache.org>
Authored: Tue Nov 22 19:45:45 2016 +0530
Committer: Brahma Reddy Battula <brahma@apache.org>
Committed: Tue Nov 22 19:45:45 2016 +0530

----------------------------------------------------------------------
 .../hadoop-common/src/main/java/overview.html   | 274 -------------------
 .../hadoop-hdfs/src/main/java/overview.html     | 274 -------------------
 pom.xml                                         |  25 --
 3 files changed, 573 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/afcf8d38/hadoop-common-project/hadoop-common/src/main/java/overview.html
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/main/java/overview.html b/hadoop-common-project/hadoop-common/src/main/java/overview.html
deleted file mode 100644
index 2c64121..0000000
--- a/hadoop-common-project/hadoop-common/src/main/java/overview.html
+++ /dev/null
@@ -1,274 +0,0 @@
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
-<html>
-<!--
-   Licensed to the Apache Software Foundation (ASF) under one or more
-   contributor license agreements.  See the NOTICE file distributed with
-   this work for additional information regarding copyright ownership.
-   The ASF licenses this file to You under the Apache License, Version 2.0
-   (the "License"); you may not use this file except in compliance with
-   the License.  You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
--->
-<head>
-   <title>Hadoop</title>
-</head>
-<body>
-
-Hadoop is a distributed computing platform.
-
-<p>Hadoop primarily consists of the <a
-href="http://hadoop.apache.org/hdfs/">Hadoop Distributed FileSystem
-(HDFS)</a> and an
-implementation of the <a href="http://hadoop.apache.org/mapreduce/">
-Map-Reduce</a> programming paradigm.</p>
-
-
-<p>Hadoop is a software framework that lets one easily write and run applications
-that process vast amounts of data. Here's what makes Hadoop especially useful:</p>
-<ul>
-  <li>
-    <b>Scalable</b>: Hadoop can reliably store and process petabytes.
-  </li>
-  <li>
-    <b>Economical</b>: It distributes the data and processing across clusters
-    of commonly available computers. These clusters can number into the thousands
-    of nodes.
-  </li>
-  <li>
-    <b>Efficient</b>: By distributing the data, Hadoop can process it in parallel
-    on the nodes where the data is located. This makes it extremely rapid.
-  </li>
-  <li>
-    <b>Reliable</b>: Hadoop automatically maintains multiple copies of data and
-    automatically redeploys computing tasks based on failures.
-  </li>
-</ul>
-
-<h2>Requirements</h2>
-
-<h3>Platforms</h3>
-
-<ul>
-  <li>
-    Hadoop has been demonstrated on GNU/Linux clusters with more than 4000 nodes.
-  </li>
-  <li>
-    Windows is also a supported platform.
-  </li>
-</ul>
-
-<h3>Requisite Software</h3>
-
-<ol>
-  <li>
-    Java 1.6.x, preferably from
-    <a href="http://java.sun.com/javase/downloads/">Sun</a>.
-    Set <tt>JAVA_HOME</tt> to the root of your Java installation.
-  </li>
-  <li>
-    ssh must be installed and sshd must be running to use Hadoop's
-    scripts to manage remote Hadoop daemons.
-  </li>
-  <li>
-    rsync may be installed to use Hadoop's scripts to manage remote
-    Hadoop installations.
-  </li>
-</ol>
-
-<h3>Installing Required Software</h3>
-
-<p>If your platform does not have the required software listed above, you
-will have to install it.</p>
-
-<p>For example on Ubuntu Linux:</p>
-<p><blockquote><pre>
-$ sudo apt-get install ssh<br>
-$ sudo apt-get install rsync<br>
-</pre></blockquote></p>
-
-<h2>Getting Started</h2>
-
-<p>First, you need to get a copy of the Hadoop code.</p>
-
-<p>Edit the file <tt>conf/hadoop-env.sh</tt> to define at least
-<tt>JAVA_HOME</tt>.</p>
-
-<p>Try the following command:</p>
-<tt>bin/hadoop</tt>
-<p>This will display the documentation for the Hadoop command script.</p>
-
-<h2>Standalone operation</h2>
-
-<p>By default, Hadoop is configured to run things in a non-distributed
-mode, as a single Java process.  This is useful for debugging, and can
-be demonstrated as follows:</p>
-<tt>
-mkdir input<br>
-cp conf/*.xml input<br>
-bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'<br>
-cat output/*
-</tt>
-<p>This will display counts for each match of the <a
-href="http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html">
-regular expression.</a></p>
-
-<p>Note that input is specified as a <em>directory</em> containing input
-files and that output is also specified as a directory where parts are
-written.</p>
-
-<h2>Distributed operation</h2>
-
-To configure Hadoop for distributed operation you must specify the
-following:
-
-<ol>
-
-<li>The NameNode (Distributed Filesystem master) host.  This is
-specified with the configuration property <tt><a
- href="../core-default.html#fs.default.name">fs.default.name</a></tt>.
-</li>
-
-<li>The org.apache.hadoop.mapred.JobTracker (MapReduce master)
-host and port.  This is specified with the configuration property
-<tt><a
-href="../mapred-default.html#mapred.job.tracker">mapred.job.tracker</a></tt>.
-</li>
-
-<li>A <em>workers</em> file that lists the names of all the hosts in
-the cluster.  The default workers file is <tt>conf/workers</tt>.
-
-</ol>
-
-<h3>Pseudo-distributed configuration</h3>
-
-You can in fact run everything on a single host.  To run things this
-way, put the following in:
-<br/>
-<br/>
-conf/core-site.xml:
-<xmp><configuration>
-
-  <property>
-    <name>fs.default.name</name>
-    <value>hdfs://localhost/</value>
-  </property>
-
-</configuration></xmp>
-
-conf/hdfs-site.xml:
-<xmp><configuration>
-
-  <property>
-    <name>dfs.replication</name>
-    <value>1</value>
-  </property>
-
-</configuration></xmp>
-
-conf/mapred-site.xml:
-<xmp><configuration>
-
-  <property>
-    <name>mapred.job.tracker</name>
-    <value>localhost:9001</value>
-  </property>
-
-</configuration></xmp>
-
-<p>(We also set the HDFS replication level to 1 in order to
-reduce warnings when running on a single node.)</p>
-
-<p>Now check that the command <br><tt>ssh localhost</tt><br>
does not
-require a password.  If it does, execute the following commands:</p>
-
-<p><tt>ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa<br>
-cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
-</tt></p>
-
-<h3>Bootstrapping</h3>
-
-<p>A new distributed filesystem must be formatted with the following
-command, run on the master node:</p>
-
-<p><tt>bin/hadoop namenode -format</tt></p>
-
-<p>The Hadoop daemons are started with the following command:</p>
-
-<p><tt>bin/start-all.sh</tt></p>
-
-<p>Daemon log output is written to the <tt>logs/</tt> directory.</p>
-
-<p>Input files are copied into the distributed filesystem as follows:</p>
-
-<p><tt>bin/hadoop fs -put input input</tt></p>
-
-<h3>Distributed execution</h3>
-
-<p>Things are run as before, but output must be copied locally to
-examine it:</p>
-
-<tt>
-bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'<br>
-bin/hadoop fs -get output output
-cat output/*
-</tt>
-
-<p>When you're done, stop the daemons with:</p>
-
-<p><tt>bin/stop-all.sh</tt></p>
-
-<h3>Fully-distributed operation</h3>
-
-<p>Fully distributed operation is just like the pseudo-distributed operation
-described above, except, specify:</p>
-
-<ol>
-
-<li>The hostname or IP address of your master server in the value
-for <tt><a
-href="../core-default.html#fs.default.name">fs.default.name</a></tt>,
-  as <tt><em>hdfs://master.example.com/</em></tt> in <tt>conf/core-site.xml</tt>.</li>
-
-<li>The host and port of the your master server in the value
-of <tt><a href="../mapred-default.html#mapred.job.tracker">mapred.job.tracker</a></tt>
-as <tt><em>master.example.com</em>:<em>port</em></tt>
in <tt>conf/mapred-site.xml</tt>.</li>
-
-<li>Directories for <tt><a
-href="../hdfs-default.html#dfs.name.dir">dfs.name.dir</a></tt> and
-<tt><a href="../hdfs-default.html#dfs.data.dir">dfs.data.dir</a>
-in <tt>conf/hdfs-site.xml</tt>.
-</tt>These are local directories used to hold distributed filesystem
-data on the master node and worker nodes respectively.  Note
-that <tt>dfs.data.dir</tt> may contain a space- or comma-separated
-list of directory names, so that data may be stored on multiple local
-devices.</li>
-
-<li><tt><a href="../mapred-default.html#mapred.local.dir">mapred.local.dir</a></tt>
-  in <tt>conf/mapred-site.xml</tt>, the local directory where temporary
-  MapReduce data is stored.  It also may be a list of directories.</li>
-
-<li><tt><a
-href="../mapred-default.html#mapred.map.tasks">mapred.map.tasks</a></tt>
-and <tt><a
-href="../mapred-default.html#mapred.reduce.tasks">mapred.reduce.tasks</a></tt>
-in <tt>conf/mapred-site.xml</tt>.
-As a rule of thumb, use 10x the
-number of worker processors for <tt>mapred.map.tasks</tt>, and 2x the
-number of worker processors for <tt>mapred.reduce.tasks</tt>.</li>
-
-</ol>
-
-<p>Finally, list all worker hostnames or IP addresses in your
-<tt>conf/workers</tt> file, one per line.  Then format your filesystem
-and start your cluster on your master node, as above.
-
-</body>
-</html>
-

http://git-wip-us.apache.org/repos/asf/hadoop/blob/afcf8d38/hadoop-hdfs-project/hadoop-hdfs/src/main/java/overview.html
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/overview.html b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/overview.html
deleted file mode 100644
index e6636d7..0000000
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/overview.html
+++ /dev/null
@@ -1,274 +0,0 @@
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
-<html>
-<!--
-   Licensed to the Apache Software Foundation (ASF) under one or more
-   contributor license agreements.  See the NOTICE file distributed with
-   this work for additional information regarding copyright ownership.
-   The ASF licenses this file to You under the Apache License, Version 2.0
-   (the "License"); you may not use this file except in compliance with
-   the License.  You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
--->
-<head>
-   <title>Hadoop</title>
-</head>
-<body>
-
-Hadoop is a distributed computing platform.
-
-<p>Hadoop primarily consists of the <a
-href="http://hadoop.apache.org/hdfs/">Hadoop Distributed FileSystem
-(HDFS)</a> and an
-implementation of the <a href="http://hadoop.apache.org/mapreduce/">
-Map-Reduce</a> programming paradigm.</p>
-
-
-<p>Hadoop is a software framework that lets one easily write and run applications
-that process vast amounts of data. Here's what makes Hadoop especially useful:</p>
-<ul>
-  <li>
-    <b>Scalable</b>: Hadoop can reliably store and process petabytes.
-  </li>
-  <li>
-    <b>Economical</b>: It distributes the data and processing across clusters
-    of commonly available computers. These clusters can number into the thousands
-    of nodes.
-  </li>
-  <li>
-    <b>Efficient</b>: By distributing the data, Hadoop can process it in parallel
-    on the nodes where the data is located. This makes it extremely rapid.
-  </li>
-  <li>
-    <b>Reliable</b>: Hadoop automatically maintains multiple copies of data and
-    automatically redeploys computing tasks based on failures.
-  </li>
-</ul>
-
-<h2>Requirements</h2>
-
-<h3>Platforms</h3>
-
-<ul>
-  <li>
-    Hadoop was been demonstrated on GNU/Linux clusters with 2000 nodes.
-  </li>
-  <li>
-    Windows is also a supported platform.
-  </li>
-</ul>
-
-<h3>Requisite Software</h3>
-
-<ol>
-  <li>
-    Java 1.6.x, preferably from
-    <a href="http://java.sun.com/javase/downloads/">Sun</a>.
-    Set <tt>JAVA_HOME</tt> to the root of your Java installation.
-  </li>
-  <li>
-    ssh must be installed and sshd must be running to use Hadoop's
-    scripts to manage remote Hadoop daemons.
-  </li>
-  <li>
-    rsync may be installed to use Hadoop's scripts to manage remote
-    Hadoop installations.
-  </li>
-</ol>
-
-<h3>Installing Required Software</h3>
-
-<p>If your platform does not have the required software listed above, you
-will have to install it.</p>
-
-<p>For example on Ubuntu Linux:</p>
-<p><blockquote><pre>
-$ sudo apt-get install ssh<br>
-$ sudo apt-get install rsync<br>
-</pre></blockquote></p>
-
-<h2>Getting Started</h2>
-
-<p>First, you need to get a copy of the Hadoop code.</p>
-
-<p>Edit the file <tt>conf/hadoop-env.sh</tt> to define at least
-<tt>JAVA_HOME</tt>.</p>
-
-<p>Try the following command:</p>
-<tt>bin/hadoop</tt>
-<p>This will display the documentation for the Hadoop command script.</p>
-
-<h2>Standalone operation</h2>
-
-<p>By default, Hadoop is configured to run things in a non-distributed
-mode, as a single Java process.  This is useful for debugging, and can
-be demonstrated as follows:</p>
-<tt>
-mkdir input<br>
-cp conf/*.xml input<br>
-bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'<br>
-cat output/*
-</tt>
-<p>This will display counts for each match of the <a
-href="http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html">
-regular expression.</a></p>
-
-<p>Note that input is specified as a <em>directory</em> containing input
-files and that output is also specified as a directory where parts are
-written.</p>
-
-<h2>Distributed operation</h2>
-
-To configure Hadoop for distributed operation you must specify the
-following:
-
-<ol>
-
-<li>The NameNode (Distributed Filesystem master) host.  This is
-specified with the configuration property <tt><a
- href="../core-default.html#fs.default.name">fs.default.name</a></tt>.
-</li>
-
-<li>The org.apache.hadoop.mapred.JobTracker (MapReduce master)
-host and port.  This is specified with the configuration property
-<tt><a
-href="../mapred-default.html#mapred.job.tracker">mapred.job.tracker</a></tt>.
-</li>
-
-<li>A <em>workers</em> file that lists the names of all the hosts in
-the cluster.  The default workers file is <tt>conf/workers</tt>.
-
-</ol>
-
-<h3>Pseudo-distributed configuration</h3>
-
-You can in fact run everything on a single host.  To run things this
-way, put the following in:
-<br/>
-<br/>
-conf/core-site.xml:
-<xmp><configuration>
-
-  <property>
-    <name>fs.default.name</name>
-    <value>hdfs://localhost/</value>
-  </property>
-
-</configuration></xmp>
-
-conf/hdfs-site.xml:
-<xmp><configuration>
-
-  <property>
-    <name>dfs.replication</name>
-    <value>1</value>
-  </property>
-
-</configuration></xmp>
-
-conf/mapred-site.xml:
-<xmp><configuration>
-
-  <property>
-    <name>mapred.job.tracker</name>
-    <value>localhost:9001</value>
-  </property>
-
-</configuration></xmp>
-
-<p>(We also set the HDFS replication level to 1 in order to
-reduce warnings when running on a single node.)</p>
-
-<p>Now check that the command <br><tt>ssh localhost</tt><br>
does not
-require a password.  If it does, execute the following commands:</p>
-
-<p><tt>ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa<br>
-cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
-</tt></p>
-
-<h3>Bootstrapping</h3>
-
-<p>A new distributed filesystem must be formatted with the following
-command, run on the master node:</p>
-
-<p><tt>bin/hadoop namenode -format</tt></p>
-
-<p>The Hadoop daemons are started with the following command:</p>
-
-<p><tt>bin/start-all.sh</tt></p>
-
-<p>Daemon log output is written to the <tt>logs/</tt> directory.</p>
-
-<p>Input files are copied into the distributed filesystem as follows:</p>
-
-<p><tt>bin/hadoop fs -put input input</tt></p>
-
-<h3>Distributed execution</h3>
-
-<p>Things are run as before, but output must be copied locally to
-examine it:</p>
-
-<tt>
-bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'<br>
-bin/hadoop fs -get output output
-cat output/*
-</tt>
-
-<p>When you're done, stop the daemons with:</p>
-
-<p><tt>bin/stop-all.sh</tt></p>
-
-<h3>Fully-distributed operation</h3>
-
-<p>Fully distributed operation is just like the pseudo-distributed operation
-described above, except, specify:</p>
-
-<ol>
-
-<li>The hostname or IP address of your master server in the value
-for <tt><a
-href="../core-default.html#fs.default.name">fs.default.name</a></tt>,
-  as <tt><em>hdfs://master.example.com/</em></tt> in <tt>conf/core-site.xml</tt>.</li>
-
-<li>The host and port of the your master server in the value
-of <tt><a href="../mapred-default.html#mapred.job.tracker">mapred.job.tracker</a></tt>
-as <tt><em>master.example.com</em>:<em>port</em></tt>
in <tt>conf/mapred-site.xml</tt>.</li>
-
-<li>Directories for <tt><a
-href="../hdfs-default.html#dfs.name.dir">dfs.name.dir</a></tt> and
-<tt><a href="../hdfs-default.html#dfs.data.dir">dfs.data.dir</a>
-in <tt>conf/hdfs-site.xml</tt>.
-</tt>These are local directories used to hold distributed filesystem
-data on the master node and worker nodes respectively.  Note
-that <tt>dfs.data.dir</tt> may contain a space- or comma-separated
-list of directory names, so that data may be stored on multiple local
-devices.</li>
-
-<li><tt><a href="../mapred-default.html#mapred.local.dir">mapred.local.dir</a></tt>
-  in <tt>conf/mapred-site.xml</tt>, the local directory where temporary
-  MapReduce data is stored.  It also may be a list of directories.</li>
-
-<li><tt><a
-href="../mapred-default.html#mapred.map.tasks">mapred.map.tasks</a></tt>
-and <tt><a
-href="../mapred-default.html#mapred.reduce.tasks">mapred.reduce.tasks</a></tt>
-in <tt>conf/mapred-site.xml</tt>.
-As a rule of thumb, use 10x the
-number of worker processors for <tt>mapred.map.tasks</tt>, and 2x the
-number of worker processors for <tt>mapred.reduce.tasks</tt>.</li>
-
-</ol>
-
-<p>Finally, list all worker hostnames or IP addresses in your
-<tt>conf/workers</tt> file, one per line.  Then format your filesystem
-and start your cluster on your master node, as above.
-
-</body>
-</html>
-

http://git-wip-us.apache.org/repos/asf/hadoop/blob/afcf8d38/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 0bdef7b..101d79f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -557,31 +557,6 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xs
     </profile>
 
     <profile>
-      <id>dist</id>
-      <!-- Profile for generating all maven artifacts and documentation. -->
-      <build>
-        <plugins>
-          <plugin>
-            <groupId>org.apache.maven.plugins</groupId>
-            <artifactId>maven-javadoc-plugin</artifactId>
-            <inherited>false</inherited>
-            <executions>
-              <execution>
-                <!-- build aggregate javadoc in parent only -->
-                <id>default-cli</id>
-                <goals>
-                  <goal>aggregate</goal>
-                </goals>
-                <configuration>
-                  <overview>hadoop-common-project/hadoop-common/src/main/java/overview.html</overview>
-                </configuration>
-              </execution>
-            </executions>
-          </plugin>
-        </plugins>
-      </build>
-    </profile>
-    <profile>
       <id>sign</id>
       <build>
         <plugins>


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message