hadoop-mapreduce-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From omal...@apache.org
Subject svn commit: r816439 [3/3] - in /hadoop/mapreduce/trunk: ./ src/docs/cn/ src/docs/src/documentation/ src/docs/src/documentation/content/xdocs/ src/docs/src/documentation/resources/images/
Date Fri, 18 Sep 2009 02:20:49 GMT
Modified: hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml?rev=816439&r1=816438&r2=816439&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml (original)
+++ hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml Fri
Sep 18 02:20:48 2009
@@ -21,7 +21,7 @@
 <document>
   
   <header>
-    <title>Map/Reduce Tutorial</title>
+    <title>MapReduce Tutorial</title>
   </header>
   
   <body>
@@ -30,22 +30,21 @@
       <title>Purpose</title>
       
       <p>This document comprehensively describes all user-facing facets of the 
-      Hadoop Map/Reduce framework and serves as a tutorial.
+      Hadoop MapReduce framework and serves as a tutorial.
       </p>
     </section>
     
     <section>
-      <title>Pre-requisites</title>
+      <title>Prerequisites</title>
       
-      <p>Ensure that Hadoop is installed, configured and is running. More
-      details:</p> 
+      <p>Make sure Hadoop is installed, configured and running. See these guides:
+      </p> 
       <ul>
         <li>
-          <a href="quickstart.html">Hadoop Quick Start</a> for first-time users.
+          <a href="http://hadoop.apache.org/common/docs/current/single_node_setup.html">Single
Node Setup</a> for first-time users.
         </li>
         <li>
-          <a href="cluster_setup.html">Hadoop Cluster Setup</a> for large, 
-          distributed clusters.
+          <a href="http://hadoop.apache.org/common/docs/current/cluster_setup.html">Cluster
Setup</a> for large, distributed clusters.
         </li>
       </ul>
     </section>
@@ -53,12 +52,12 @@
     <section>
       <title>Overview</title>
       
-      <p>Hadoop Map/Reduce is a software framework for easily writing 
+      <p>Hadoop MapReduce is a software framework for easily writing 
       applications which process vast amounts of data (multi-terabyte data-sets) 
       in-parallel on large clusters (thousands of nodes) of commodity 
       hardware in a reliable, fault-tolerant manner.</p>
       
-      <p>A Map/Reduce <em>job</em> usually splits the input data-set into

+      <p>A MapReduce <em>job</em> usually splits the input data-set into

       independent chunks which are processed by the <em>map tasks</em> in a
       completely parallel manner. The framework sorts the outputs of the maps, 
       which are then input to the <em>reduce tasks</em>. Typically both the 
@@ -67,13 +66,14 @@
       tasks.</p>
       
       <p>Typically the compute nodes and the storage nodes are the same, that is, 
-      the Map/Reduce framework and the Hadoop Distributed File System (see <a href="hdfs_design.html">HDFS
Architecture </a>) 
+      the MapReduce framework and the 
+      <a href="http://hadoop.apache.org/hdfs/docs/current/index.html">Hadoop Distributed
File System</a> (HDFS) 
       are running on the same set of nodes. This configuration
       allows the framework to effectively schedule tasks on the nodes where data 
       is already present, resulting in very high aggregate bandwidth across the 
       cluster.</p>
       
-      <p>The Map/Reduce framework consists of a single master 
+      <p>The MapReduce framework consists of a single master 
       <code>JobTracker</code> and one slave <code>TaskTracker</code>
per 
       cluster-node. The master is responsible for scheduling the jobs' component 
       tasks on the slaves, monitoring them and re-executing the failed tasks. The 
@@ -90,7 +90,7 @@
       information to the job-client.</p>
       
       <p>Although the Hadoop framework is implemented in Java<sup>TM</sup>,

-      Map/Reduce applications need not be written in Java.</p>
+      MapReduce applications need not be written in Java.</p>
       <ul>
         <li>
           <a href="ext:api/org/apache/hadoop/streaming/package-summary">
@@ -101,7 +101,7 @@
         <li>
           <a href="ext:api/org/apache/hadoop/mapred/pipes/package-summary">
           Hadoop Pipes</a> is a <a href="http://www.swig.org/">SWIG</a>-
-          compatible <em>C++ API</em> to implement Map/Reduce applications (non

+          compatible <em>C++ API</em> to implement MapReduce applications (non

           JNI<sup>TM</sup> based).
         </li>
       </ul>
@@ -110,7 +110,7 @@
     <section>
       <title>Inputs and Outputs</title>
 
-      <p>The Map/Reduce framework operates exclusively on 
+      <p>The MapReduce framework operates exclusively on 
       <code>&lt;key, value&gt;</code> pairs, that is, the framework views
the 
       input to the job as a set of <code>&lt;key, value&gt;</code> pairs
and 
       produces a set of <code>&lt;key, value&gt;</code> pairs as the
output of 
@@ -124,7 +124,7 @@
       WritableComparable</a> interface to facilitate sorting by the framework.
       </p>
 
-      <p>Input and Output types of a Map/Reduce job:</p>
+      <p>Input and Output types of a MapReduce job:</p>
       <p>
         (input) <code>&lt;k1, v1&gt;</code> 
         -&gt; 
@@ -145,14 +145,16 @@
     <section>
       <title>Example: WordCount v1.0</title>
       
-      <p>Before we jump into the details, lets walk through an example Map/Reduce 
+      <p>Before we jump into the details, lets walk through an example MapReduce 
       application to get a flavour for how they work.</p>
       
       <p><code>WordCount</code> is a simple application that counts the
number of
       occurences of each word in a given input set.</p>
       
-      <p>This works with a local-standalone, pseudo-distributed or fully-distributed

-      Hadoop installation(see <a href="quickstart.html"> Hadoop Quick Start</a>).</p>
+      <p>This example works with a 
+      pseudo-distributed (<a href="http://hadoop.apache.org/common/docs/current/single_node_setup.html#SingleNodeSetup">Single
Node Setup</a>) 
+     or fully-distributed (<a href="http://hadoop.apache.org/common/docs/current/cluster_setup.html">Cluster
Setup</a>) 
+      Hadoop installation.</p>   
       
       <section>
         <title>Source Code</title>
@@ -609,7 +611,7 @@
         as arguments that are unzipped/unjarred and a link with name of the
         jar/zip are created in the current working directory of tasks. More
         details about the command line options are available at 
-        <a href="commands_manual.html"> Hadoop Command Guide.</a></p>
+        <a href="commands_manual.html"> Hadoop Commands Guide.</a></p>
         
         <p>Running <code>wordcount</code> example with 
         <code>-libjars</code> and <code>-files</code>:<br/>
@@ -697,10 +699,10 @@
     </section>
     
     <section>
-      <title>Map/Reduce - User Interfaces</title>
+      <title>MapReduce - User Interfaces</title>
       
       <p>This section provides a reasonable amount of detail on every user-facing 
-      aspect of the Map/Reduce framwork. This should help users implement, 
+      aspect of the MapReduce framwork. This should help users implement, 
       configure and tune their jobs in a fine-grained manner. However, please 
       note that the javadoc for each class/interface remains the most 
       comprehensive documentation available; this is only meant to be a tutorial.
@@ -739,7 +741,7 @@
           to be of the same type as the input records. A given input pair may 
           map to zero or many output pairs.</p> 
  
-          <p>The Hadoop Map/Reduce framework spawns one map task for each 
+          <p>The Hadoop MapReduce framework spawns one map task for each 
           <code>InputSplit</code> generated by the <code>InputFormat</code>
for 
           the job.</p>
           
@@ -950,7 +952,7 @@
           <title>Reporter</title>
         
           <p><a href="ext:api/org/apache/hadoop/mapred/reporter">
-          Reporter</a> is a facility for Map/Reduce applications to report 
+          Reporter</a> is a facility for MapReduce applications to report 
           progress, set application-level status messages and update 
           <code>Counters</code>.</p>
  
@@ -973,12 +975,12 @@
         
           <p><a href="ext:api/org/apache/hadoop/mapred/outputcollector">
           OutputCollector</a> is a generalization of the facility provided by
-          the Map/Reduce framework to collect data output by the 
+          the MapReduce framework to collect data output by the 
           <code>Mapper</code> or the <code>Reducer</code> (either
the 
           intermediate outputs or the output of the job).</p>
         </section>
       
-        <p>Hadoop Map/Reduce comes bundled with a 
+        <p>Hadoop MapReduce comes bundled with a 
         <a href="ext:api/org/apache/hadoop/mapred/lib/package-summary">
         library</a> of generally useful mappers, reducers, and partitioners.</p>
       </section>
@@ -987,10 +989,10 @@
         <title>Job Configuration</title>
         
         <p><a href="ext:api/org/apache/hadoop/mapred/jobconf">
-        JobConf</a> represents a Map/Reduce job configuration.</p>
+        JobConf</a> represents a MapReduce job configuration.</p>
  
         <p><code>JobConf</code> is the primary interface for a user to
describe
-        a Map/Reduce job to the Hadoop framework for execution. The framework 
+        a MapReduce job to the Hadoop framework for execution. The framework 
         tries to faithfully execute the job as described by <code>JobConf</code>,

         however:</p> 
         <ul>
@@ -1058,7 +1060,7 @@
         <code>-Djava.library.path=&lt;&gt;</code> etc. If the 
         <code>mapred.{map|reduce}.child.java.opts</code> parameters contains
the 
         symbol <em>@taskid@</em> it is interpolated with value of 
-        <code>taskid</code> of the map/reduce task.</p>
+        <code>taskid</code> of the MapReduce task.</p>
         
         <p>Here is an example with multiple arguments and substitutions, 
         showing jvm GC logging, and start of a passwordless JVM JMX agent so that
@@ -1109,9 +1111,9 @@
         
         <p>Note: <code>mapred.{map|reduce}.child.java.opts</code> are used
only 
         for configuring the launched child tasks from task tracker. Configuring 
-        the memory options for daemons is documented in 
-        <a href="cluster_setup.html#Configuring+the+Environment+of+the+Hadoop+Daemons">
-        cluster_setup.html </a></p>
+        the memory options for daemons is documented under
+        <a href="http://hadoop.apache.org/common/docs/current/cluster_setup.html#Configuring+the+Environment+of+the+Hadoop+Daemons">
+        Configuring the Environment of the Hadoop Daemons</a> (Cluster Setup).</p>
         
         <p>The memory available to some parts of the framework is also
         configurable. In map and reduce tasks, performance may be influenced
@@ -1428,9 +1430,9 @@
         System.loadLibrary</a> or 
         <a href="http://java.sun.com/javase/6/docs/api/java/lang/System.html#load(java.lang.String)">
         System.load</a>. More details on how to load shared libraries through 
-        distributed cache are documented at 
-        <a href="native_libraries.html#Loading+native+libraries+through+DistributedCache">
-        native_libraries.html</a></p>
+        distributed cache are documented under 
+        <a href="http://hadoop.apache.org/common/docs/current/native_libraries.html#Loading+Native+Libraries+Through+DistributedCache">
+        Building Native Hadoop Libraries</a>.</p>
         </section>
       </section>
       
@@ -1442,7 +1444,7 @@
         with the <code>JobTracker</code>.</p>
  
         <p><code>JobClient</code> provides facilities to submit jobs, track
their 
-        progress, access component-tasks' reports and logs, get the Map/Reduce 
+        progress, access component-tasks' reports and logs, get the MapReduce 
         cluster's status information and so on.</p>
  
         <p>The job submission process involves:</p>
@@ -1454,7 +1456,7 @@
             <code>DistributedCache</code> of the job, if necessary.
           </li>
           <li>
-            Copying the job's jar and configuration to the Map/Reduce system 
+            Copying the job's jar and configuration to the MapReduce system 
             directory on the <code>FileSystem</code>.
           </li>
           <li>
@@ -1484,8 +1486,8 @@
         <section>
           <title>Job Control</title>
  
-          <p>Users may need to chain Map/Reduce jobs to accomplish complex
-          tasks which cannot be done via a single Map/Reduce job. This is fairly
+          <p>Users may need to chain MapReduce jobs to accomplish complex
+          tasks which cannot be done via a single MapReduce job. This is fairly
           easy since the output of the job typically goes to distributed 
           file-system, and the output, in turn, can be used as the input for the 
           next job.</p>
@@ -1519,10 +1521,10 @@
         <title>Job Input</title>
         
         <p><a href="ext:api/org/apache/hadoop/mapred/inputformat">
-        InputFormat</a> describes the input-specification for a Map/Reduce job.
+        InputFormat</a> describes the input-specification for a MapReduce job.
         </p> 
  
-        <p>The Map/Reduce framework relies on the <code>InputFormat</code>
of 
+        <p>The MapReduce framework relies on the <code>InputFormat</code>
of 
         the job to:</p>
         <ol>
           <li>Validate the input-specification of the job.</li>
@@ -1601,10 +1603,10 @@
         <title>Job Output</title>
         
         <p><a href="ext:api/org/apache/hadoop/mapred/outputformat">
-        OutputFormat</a> describes the output-specification for a Map/Reduce 
+        OutputFormat</a> describes the output-specification for a MapReduce 
         job.</p>
 
-        <p>The Map/Reduce framework relies on the <code>OutputFormat</code>
of 
+        <p>The MapReduce framework relies on the <code>OutputFormat</code>
of 
         the job to:</p>
         <ol>
           <li>
@@ -1645,9 +1647,9 @@
         
         <p><a href="ext:api/org/apache/hadoop/mapred/outputcommitter">
         OutputCommitter</a> describes the commit of task output for a 
-        Map/Reduce job.</p>
+        MapReduce job.</p>
 
-        <p>The Map/Reduce framework relies on the <code>OutputCommitter</code>
+        <p>The MapReduce framework relies on the <code>OutputCommitter</code>
         of the job to:</p>
         <ol>
           <li>
@@ -1705,7 +1707,7 @@
           (using the attemptid, say <code>attempt_200709221812_0001_m_000000_0</code>),

           not just per task.</p> 
  
-          <p>To avoid these issues the Map/Reduce framework, when the 
+          <p>To avoid these issues the MapReduce framework, when the 
           <code>OutputCommitter</code> is <code>FileOutputCommitter</code>,

           maintains a special 
           <code>${mapred.output.dir}/_temporary/_${taskid}</code> sub-directory
@@ -1729,10 +1731,10 @@
           <p>Note: The value of <code>${mapred.work.output.dir}</code>
during 
           execution of a particular task-attempt is actually 
           <code>${mapred.output.dir}/_temporary/_{$taskid}</code>, and this value
is 
-          set by the Map/Reduce framework. So, just create any side-files in the 
+          set by the MapReduce framework. So, just create any side-files in the 
           path  returned by
           <a href="ext:api/org/apache/hadoop/mapred/fileoutputformat/getworkoutputpath">
-          FileOutputFormat.getWorkOutputPath() </a>from Map/Reduce 
+          FileOutputFormat.getWorkOutputPath() </a>from MapReduce 
           task to take advantage of this feature.</p>
           
           <p>The entire discussion holds true for maps of jobs with 
@@ -1781,7 +1783,7 @@
           <title>Counters</title>
           
           <p><code>Counters</code> represent global counters, defined either
by 
-          the Map/Reduce framework or applications. Each <code>Counter</code>
can 
+          the MapReduce framework or applications. Each <code>Counter</code>
can 
           be of any <code>Enum</code> type. Counters of a particular 
           <code>Enum</code> are bunched into groups of type 
           <code>Counters.Group</code>.</p>
@@ -1805,7 +1807,7 @@
           files efficiently.</p>
  
           <p><code>DistributedCache</code> is a facility provided by the

-          Map/Reduce framework to cache files (text, archives, jars and so on) 
+          MapReduce framework to cache files (text, archives, jars and so on) 
           needed by applications.</p>
  
           <p>Applications specify the files to be cached via urls (hdfs://)
@@ -1929,7 +1931,7 @@
           interface supports the handling of generic Hadoop command-line options.
           </p>
           
-          <p><code>Tool</code> is the standard for any Map/Reduce tool
or 
+          <p><code>Tool</code> is the standard for any MapReduce tool or

           application. The application should delegate the handling of 
           standard command-line options to 
           <a href="ext:api/org/apache/hadoop/util/genericoptionsparser">
@@ -1962,7 +1964,7 @@
           <title>IsolationRunner</title>
           
           <p><a href="ext:api/org/apache/hadoop/mapred/isolationrunner">
-          IsolationRunner</a> is a utility to help debug Map/Reduce programs.</p>
+          IsolationRunner</a> is a utility to help debug MapReduce programs.</p>
           
           <p>To use the <code>IsolationRunner</code>, first set 
           <code>keep.failed.tasks.files</code> to <code>true</code>

@@ -2000,7 +2002,7 @@
           <p>Once user configures that profiling is needed, she/he can use
           the configuration property 
           <code>mapred.task.profile.{maps|reduces}</code> to set the ranges
-          of Map/Reduce tasks to profile. The value can be set using the api 
+          of MapReduce tasks to profile. The value can be set using the api 
           <a href="ext:api/org/apache/hadoop/mapred/jobconf/setprofiletaskrange">
           JobConf.setProfileTaskRange(boolean,String)</a>.
           By default, the specified range is <code>0-2</code>.</p>
@@ -2021,8 +2023,8 @@
         
         <section>
           <title>Debugging</title>
-          <p>The Map/Reduce framework provides a facility to run user-provided 
-          scripts for debugging. When a Map/Reduce task fails, a user can run 
+          <p>The MapReduce framework provides a facility to run user-provided 
+          scripts for debugging. When a MapReduce task fails, a user can run 
           a debug script, to process task logs for example. The script is 
           given access to the task's stdout and stderr outputs, syslog and 
           jobconf. The output from the debug script's stdout and stderr is 
@@ -2055,7 +2057,7 @@
             
           <p>The arguments to the script are the task's stdout, stderr, 
           syslog and jobconf files. The debug command, run on the node where
-          the Map/Reduce task failed, is: <br/>
+          the MapReduce task failed, is: <br/>
           <code> $script $stdout $stderr $syslog $jobconf </code> </p>

 
           <p> Pipes programs have the c++ program name as a fifth argument
@@ -2075,14 +2077,14 @@
           <title>JobControl</title>
           
           <p><a href="ext:api/org/apache/hadoop/mapred/jobcontrol/package-summary">
-          JobControl</a> is a utility which encapsulates a set of Map/Reduce jobs
+          JobControl</a> is a utility which encapsulates a set of MapReduce jobs
           and their dependencies.</p>
         </section>
         
         <section>
           <title>Data Compression</title>
           
-          <p>Hadoop Map/Reduce provides facilities for the application-writer to
+          <p>Hadoop MapReduce provides facilities for the application-writer to
           specify compression for both intermediate map-outputs and the
           job-outputs i.e. output of the reduces. It also comes bundled with
           <a href="ext:api/org/apache/hadoop/io/compress/compressioncodec">
@@ -2091,10 +2093,11 @@
           algorithm. The <a href="ext:gzip">gzip</a> file format is also
           supported.</p>
           
-          <p>Hadoop also provides native implementations of the above compression
+         <p>Hadoop also provides native implementations of the above compression
           codecs for reasons of both performance (zlib) and non-availability of
-          Java libraries. More details on their usage and availability are
-          available <a href="native_libraries.html">here</a>.</p>
+          Java libraries. For more information see the
+          <a href="http://hadoop.apache.org/common/docs/current/native_libraries.html">Native
Libraries Guide</a>.</p>
+          
           
           <section>
             <title>Intermediate Outputs</title>
@@ -2211,13 +2214,13 @@
       <title>Example: WordCount v2.0</title>
       
       <p>Here is a more complete <code>WordCount</code> which uses many
of the
-      features provided by the Map/Reduce framework we discussed so far.</p>
+      features provided by the MapReduce framework we discussed so far.</p>
       
-      <p>This needs the HDFS to be up and running, especially for the 
+      <p>This example needs the HDFS to be up and running, especially for the 
       <code>DistributedCache</code>-related features. Hence it only works with
a 
-      <a href="quickstart.html#SingleNodeSetup">pseudo-distributed</a> or
-      <a href="quickstart.html#Fully-Distributed+Operation">fully-distributed</a>

-      Hadoop installation.</p>      
+      pseudo-distributed (<a href="http://hadoop.apache.org/common/docs/current/single_node_setup.html#SingleNodeSetup">Single
Node Setup</a>) 
+     or fully-distributed (<a href="http://hadoop.apache.org/common/docs/current/cluster_setup.html#Fully-Distributed+Operation">Cluster
Setup</a>) 
+      Hadoop installation.</p>     
       
       <section>
         <title>Source Code</title>
@@ -3163,7 +3166,7 @@
         <title>Highlights</title>
         
         <p>The second version of <code>WordCount</code> improves upon the

-        previous one by using some features offered by the Map/Reduce framework:
+        previous one by using some features offered by the MapReduce framework:
         </p>
         <ul>
           <li>

Modified: hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/site.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/site.xml?rev=816439&r1=816438&r2=816439&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/site.xml (original)
+++ hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/site.xml Fri Sep 18 02:20:48
2009
@@ -34,39 +34,22 @@
   
    <docs label="Getting Started"> 
 		<overview   				label="Overview" 					href="index.html" />
-		<quickstart 				label="Quick Start"        		href="quickstart.html" />
-		<setup     					label="Cluster Setup"      		href="cluster_setup.html" />
-		<mapred    				label="Map/Reduce Tutorial" 	href="mapred_tutorial.html" />
-  </docs>	
+		<mapred    				label="MapReduce Tutorial" 	href="mapred_tutorial.html" />
+		 <streaming 				label="Hadoop Streaming"  href="streaming.html" />
+   </docs>	
 		
- <docs label="Programming Guides">
-		<commands 				label="Commands"     					href="commands_manual.html" />
-		<distcp    					label="DistCp"       						href="distcp.html" />
-		<native_lib    				label="Native Libraries" 					href="native_libraries.html" />
-		<streaming 				label="Streaming"          				href="streaming.html" />
-		<fair_scheduler 			label="Fair Scheduler" 					href="fair_scheduler.html"/>
-		<cap_scheduler 		label="Capacity Scheduler" 			href="capacity_scheduler.html"/>
-		<SLA					 	label="Service Level Authorization" 	href="service_level_auth.html"/>
-		<vaidya    					label="Vaidya" 								href="vaidya.html"/>
-		<archives  				label="Archives"     						href="hadoop_archives.html"/>
+  <docs label="Guides">
+		<commands 				label="Hadoop Commands"  href="commands_manual.html" />
+		<distcp    					label="DistCp"       href="distcp.html" />
+		<vaidya    					label="Vaidya" 		href="vaidya.html"/>
+		<archives  				label="Hadoop Archives"     href="hadoop_archives.html"/>
    </docs>
    
-   <docs label="HDFS">
-		<hdfs_user      				label="User Guide"    							href="hdfs_user_guide.html" />
-		<hdfs_arch     				label="Architecture"  								href="hdfs_design.html" />	
-		<hdfs_fs       	 				label="File System Shell Guide"     		href="hdfs_shell.html" />
-		<hdfs_perm      				label="Permissions Guide"    					href="hdfs_permissions_guide.html"
/>
-		<hdfs_quotas     			label="Quotas Guide" 							href="hdfs_quota_admin_guide.html" />
-		<hdfs_SLG        			label="Synthetic Load Generator Guide"  href="SLG_user_guide.html"
/>
-		<hdfs_imageviewer						label="Offline Image Viewer Guide"	href="hdfs_imageviewer.html"
/>
-		<hdfs_libhdfs   				label="C API libhdfs"         						href="libhdfs.html" /> 
-   </docs> 
-   
-   <docs label="HOD">
-		<hod_user 	label="User Guide" 	href="hod_user_guide.html"/>
-		<hod_admin 	label="Admin Guide" 	href="hod_admin_guide.html"/>
-		<hod_config 	label="Config Guide" 	href="hod_config_guide.html"/> 
-   </docs> 
+    <docs label="Schedulers">
+        <cap_scheduler 		label="Capacity Scheduler"     href="capacity_scheduler.html"/>
+		<fair_scheduler 			label="Fair Scheduler"            href="fair_scheduler.html"/>
+		<cap_scheduler 		label="Hod Scheduler" 			href="hod_scheduler.html"/>
+    </docs>
    
    <docs label="Miscellaneous"> 
 		<api       	label="API Docs"           href="ext:api/index" />
@@ -78,19 +61,20 @@
    </docs> 
    
   <external-refs>
-    <site      href="http://hadoop.apache.org/core/"/>
-    <lists     href="http://hadoop.apache.org/core/mailing_lists.html"/>
-    <archive   href="http://mail-archives.apache.org/mod_mbox/hadoop-core-commits/"/>
-    <releases  href="http://hadoop.apache.org/core/releases.html">
-      <download href="#Download" />
+    <site      href="http://hadoop.apache.org/mapreduce/"/>
+    <lists     href="http://hadoop.apache.org/mapreduce/mailing_lists.html"/>
+    <archive   href="http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-commits/"/>
+    <releases  href="http://hadoop.apache.org/mapreduce/releases.html">
+           <download href="#Download" />
     </releases>
-    <jira      href="http://hadoop.apache.org/core/issue_tracking.html"/>
-    <wiki      href="http://wiki.apache.org/hadoop/" />
-    <faq       href="http://wiki.apache.org/hadoop/FAQ" />
-    <hadoop-default href="http://hadoop.apache.org/core/docs/current/hadoop-default.html"
/>
-    <core-default href="http://hadoop.apache.org/core/docs/current/core-default.html"
/>
-    <hdfs-default href="http://hadoop.apache.org/core/docs/current/hdfs-default.html"
/>
-    <mapred-default href="http://hadoop.apache.org/core/docs/current/mapred-default.html"
/>
+    <jira      href="http://hadoop.apache.org/mapreduce/issue_tracking.html"/>
+    <wiki      href="http://wiki.apache.org/hadoop/MapReduce" />
+    <faq       href="http://wiki.apache.org/hadoop/MapReduce/FAQ" />
+    
+    <common-default href="http://hadoop.apache.org/common/docs/current/common-default.html"
/>
+    <hdfs-default href="http://hadoop.apache.org/hdfs/docs/current/hdfs-default.html"
/>
+    <mapred-default href="http://hadoop.apache.org/mapreduce/docs/current/mapred-default.html"
/>
+    
     <zlib      href="http://www.zlib.net/" />
     <gzip      href="http://www.gzip.org/" />
     <bzip      href="http://www.bzip.org/" />

Modified: hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/streaming.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/streaming.xml?rev=816439&r1=816438&r2=816439&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/streaming.xml (original)
+++ hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/streaming.xml Fri Sep
18 02:20:48 2009
@@ -30,7 +30,7 @@
 <title>Hadoop Streaming</title>
 
 <p>
-Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows
you to create and run Map/Reduce jobs with any executable or 
+Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows
you to create and run MapReduce jobs with any executable or 
 script as the mapper and/or the reducer. For example:
 </p>
 <source>
@@ -47,7 +47,7 @@
 <title>How Streaming Works </title>
 <p>
 In the above example, both the mapper and the reducer are executables that read the input
from stdin (line by line) and emit the output to stdout. 
-The utility will create a Map/Reduce job, submit the job to an appropriate cluster, and monitor
the progress of the job until it completes.
+The utility will create a MapReduce job, submit the job to an appropriate cluster, and monitor
the progress of the job until it completes.
 </p>
 <p>
   When an executable is specified for mappers, each mapper task will launch the executable
as a separate process when the mapper is initialized. 
@@ -63,7 +63,7 @@
 prefix of a line up to the first tab character is the key and the rest of the line (excluding
the tab character) is the value. However, this can be customized, as discussed later.
 </p>
 <p>
-This is the basis for the communication protocol between the Map/Reduce framework and the
streaming mapper/reducer.
+This is the basis for the communication protocol between the MapReduce framework and the
streaming mapper/reducer.
 </p>
 <p>
 You can supply a Java class as the mapper and/or the reducer. The above example is equivalent
to:
@@ -161,7 +161,7 @@
 <section>
 <title>Specifying Other Plugins for Jobs </title>
 <p>
-Just as with a normal Map/Reduce job, you can specify other plugins for a streaming job:
+Just as with a normal MapReduce job, you can specify other plugins for a streaming job:
 </p>
 <source>
    -inputformat JavaClassName
@@ -188,7 +188,7 @@
 <!-- GENERIC COMMAND OPTIONS-->
 <section>
 <title>Generic Command Options</title>
-<p>Streaming supports <a href="streaming.html#Streaming+Command+Options">streaming
command options</a> as well as generic command options.
+<p>Streaming supports generic command options as well as <a href="streaming.html#Streaming+Command+Options">streaming
command options</a>.
 The general command line syntax is shown below. </p>
 <p><strong>Note:</strong> Be sure to place the generic options before the
streaming options, otherwise the command will fail. 
 For an example, see <a href="streaming.html#Making+Archives+Available+to+Tasks">Making
Archives Available to Tasks</a>.</p>
@@ -201,7 +201,7 @@
 <tr><td> -D  property=value </td><td> Optional </td><td>
Use value for given property </td></tr>
 <tr><td> -fs host:port or local </td><td> Optional </td><td>
Specify a namenode </td></tr>
 <tr><td> -jt host:port or local </td><td> Optional </td><td>
Specify a job tracker </td></tr>
-<tr><td> -files </td><td> Optional </td><td> Specify
comma-separated files to be copied to the Map/Reduce cluster </td></tr>
+<tr><td> -files </td><td> Optional </td><td> Specify
comma-separated files to be copied to the MapReduce cluster </td></tr>
 <tr><td> -libjars  </td><td> Optional </td><td> Specify
comma-separated jar files to include in the classpath </td></tr>
 <tr><td> -archives </td><td> Optional </td><td> Specify
comma-separated archives to be unarchived on the compute machines </td></tr>
 </table>
@@ -235,7 +235,7 @@
 <title>Specifying Map-Only Jobs </title>
 <p>
 Often, you may want to process input data using a map function only. To do this, simply set
mapred.reduce.tasks to zero. 
-The Map/Reduce framework will not create any reducer tasks. Rather, the outputs of the mapper
tasks will be the final output of the job.
+The MapReduce framework will not create any reducer tasks. Rather, the outputs of the mapper
tasks will be the final output of the job.
 </p>
 <source>
     -D mapred.reduce.tasks=0
@@ -263,7 +263,7 @@
 <section>
 <title>Customizing How Lines are Split into Key/Value Pairs</title>
 <p>
-As noted earlier, when the Map/Reduce framework reads a line from the stdout of the mapper,
it splits the line into a key/value pair. 
+As noted earlier, when the MapReduce framework reads a line from the stdout of the mapper,
it splits the line into a key/value pair. 
 By default, the prefix of the line up to the first tab character is the key and the rest
of the line (excluding the tab character) is the value.
 </p>
 <p>
@@ -290,7 +290,7 @@
 Similarly, you can use "-D stream.reduce.output.field.separator=SEP" and "-D stream.num.reduce.output.fields=NUM"
to specify 
 the nth field separator in a line of the reduce outputs as the separator between the key
and the value.
 </p>
-<p> Similarly, you can specify "stream.map.input.field.separator" and "stream.reduce.input.field.separator"
as the input separator for Map/Reduce 
+<p> Similarly, you can specify "stream.map.input.field.separator" and "stream.reduce.input.field.separator"
as the input separator for MapReduce 
 inputs. By default the separator is the tab character.</p>
 </section>
 
@@ -306,8 +306,7 @@
 <p><strong>Note:</strong>
 The -files and -archives options are generic options.
 Be sure to place the generic options before the command options, otherwise the command will
fail. 
-For an example, see <a href="streaming.html#The+-archives+Option">The -archives Option</a>.
-Also see <a href="streaming.html#Other+Supported+Options">Other Supported Options</a>.
+For an example, see <a href="streaming.html#Making+Archives+Available+to+Tasks">Making
Archives Available to Tasks</a>.
 </p>
 
 <section>
@@ -401,7 +400,7 @@
 <p>
 Hadoop has a library class, 
 <a href="ext:api/org/apache/hadoop/mapred/lib/keyfieldbasedpartitioner">KeyFieldBasedPartitioner</a>,

-that is useful for many applications. This class allows the Map/Reduce 
+that is useful for many applications. This class allows the MapReduce 
 framework to partition the map outputs based on certain key fields, not
 the whole keys. For example:
 </p>
@@ -421,8 +420,8 @@
 <p>
 Here, <em>-D stream.map.output.field.separator=.</em> and <em>-D stream.num.map.output.key.fields=4</em>
are as explained in previous example. The two variables are used by streaming to identify
the key/value pair of mapper. 
 </p><p>
-The map output keys of the above Map/Reduce job normally have four fields
-separated by ".". However, the Map/Reduce framework will partition the map
+The map output keys of the above MapReduce job normally have four fields
+separated by ".". However, the MapReduce framework will partition the map
 outputs by the first two fields of the keys using the 
 <em>-D mapred.text.key.partitioner.options=-k1,2</em> option. 
 Here, <em>-D map.output.key.field.separator=.</em> specifies the separator 
@@ -482,8 +481,8 @@
     -reducer org.apache.hadoop.mapred.lib.IdentityReducer 
 </source>
 <p>
-The map output keys of the above Map/Reduce job normally have four fields
-separated by ".". However, the Map/Reduce framework will sort the 
+The map output keys of the above MapReduce job normally have four fields
+separated by ".". However, the MapReduce framework will sort the 
 outputs by the second field of the keys using the 
 <em>-D mapred.text.key.comparator.options=-k2,2nr</em> option. 
 Here, <em>-n</em> specifies that the sorting is numerical sorting and 
@@ -653,7 +652,7 @@
 <section>
 <title>How many reducers should I use? </title>
 <p>
-See the Hadoop Wiki for details: <a href="mapred_tutorial.html#Reducer">Reducer</a>
+For details see <a href="mapred_tutorial.html#Reducer">Reducer</a>.
 </p>
 </section>
 
@@ -790,7 +789,7 @@
 <section>
 <title>How do I get the JobConf variables in a streaming job's mapper/reducer?</title>
 <p>
-See <a href="mapred_tutorial.html#Configured+Parameters">Configured Parameters</a>.

+See the <a href="mapred_tutorial.html#Configured+Parameters">Configured Parameters</a>.

 During the execution of a streaming job, the names of the "mapred" parameters are transformed.
The dots ( . ) become underscores ( _ ).
 For example, mapred.job.id becomes mapred_job_id and mapred.jar becomes mapred_jar. In your
code, use the parameter names with the underscores.
 </p>

Modified: hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/tabs.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/tabs.xml?rev=816439&r1=816438&r2=816439&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/tabs.xml (original)
+++ hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/tabs.xml Fri Sep 18 02:20:48
2009
@@ -30,8 +30,8 @@
     directory (ends in '/'), in which case /index.html will be added
   -->
 
-  <tab label="Project" href="http://hadoop.apache.org/core/" />
-  <tab label="Wiki" href="http://wiki.apache.org/hadoop" />
-  <tab label="Hadoop 0.21 Documentation" dir="" />  
+  <tab label="Project" href="http://hadoop.apache.org/mapreduce/" />
+  <tab label="Wiki" href="http://wiki.apache.org/hadoop/MapReduce" />
+  <tab label="MapReduce 0.21 Documentation" dir="" />  
   
 </tabs>

Modified: hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/vaidya.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/vaidya.xml?rev=816439&r1=816438&r2=816439&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/vaidya.xml (original)
+++ hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/vaidya.xml Fri Sep 18
02:20:48 2009
@@ -29,8 +29,8 @@
     <section>
       <title>Purpose</title>
       
-      <p>This document describes various user-facing facets of Hadoop Vaidya, a performance
diagnostic tool for map/reduce jobs. It
-         describes how to execute a default set of rules against your map/reduce job counters
and
+      <p>This document describes various user-facing facets of Hadoop Vaidya, a performance
diagnostic tool for MapReduce jobs. It
+         describes how to execute a default set of rules against your MapReduce job counters
and
          how to write and execute new rules to detect specific performance problems. 
       </p>
       <p>A few sample test rules are provided with the tool with the objective of growing
the rules database over the time. 
@@ -41,7 +41,7 @@
     </section>
     
     <section>
-      <title>Pre-requisites</title>
+      <title>Prerequisites</title>
       
       <p>Ensure that Hadoop is installed and configured. More details:</p> 
       <ul>
@@ -59,11 +59,11 @@
       
       <p>Hadoop Vaidya (Vaidya in Sanskrit language means "one who knows", or "a physician")

 	    is a rule based performance diagnostic tool for 
-        Map/Reduce jobs. It performs a post execution analysis of map/reduce 
+        MapReduce jobs. It performs a post execution analysis of MapReduce 
         job by parsing and collecting execution statistics through job history 
         and job configuration files. It runs a set of predefined tests/rules 
         against job execution statistics to diagnose various performance problems. 
-        Each test rule detects a specific performance problem with the Map/Reduce job and
provides 
+        Each test rule detects a specific performance problem with the MapReduce job and
provides 
         a targeted advice to the user. This tool generates an XML report based on 
         the evaluation results of individual test rules.
       </p>
@@ -75,9 +75,9 @@
 	 
 	 <p> This section describes main concepts and terminology involved with Hadoop Vaidya,</p>
 		<ul>
-			<li> <em>PostExPerformanceDiagnoser</em>: This class extends the base
Diagnoser class and acts as a driver for post execution performance analysis of Map/Reduce
Jobs. 
+			<li> <em>PostExPerformanceDiagnoser</em>: This class extends the base
Diagnoser class and acts as a driver for post execution performance analysis of MapReduce
Jobs. 
                        It detects performance inefficiencies by executing a set of performance
diagnosis rules against the job execution statistics.</li>
-			<li> <em>Job Statistics</em>: This includes the job configuration information
(job.xml) and various counters logged by Map/Reduce job as a part of the job history log
+			<li> <em>Job Statistics</em>: This includes the job configuration information
(job.xml) and various counters logged by MapReduce job as a part of the job history log
 		           file. The counters are parsed and collected into the Job Statistics data structures,
which contains global job level aggregate counters and 
 			     a set of counters for each Map and Reduce task.</li>
 			<li> <em>Diagnostic Test/Rule</em>: This is a program logic that detects
the inefficiency of M/R job based on the job statistics. The
@@ -139,10 +139,10 @@
 	</section>
 	
     <section>
-		<title>How to Write and Execute your own Tests</title>
+		<title>How to Write and Execute Your Own Tests</title>
 		<p>Writing and executing your own test rules is not very hard. You can take a look
at Hadoop Vaidya source code for existing set of tests. 
-		   The source code is at this <a href="http://svn.apache.org/viewvc/hadoop/core/trunk/src/contrib/vaidya/src/java/org/apache/hadoop/vaidya/">hadoop
svn repository location</a>
-		   . The default set of tests are under <code>"postexdiagnosis/tests/"</code>
folder.</p>
+		   The source code is at this <a href="http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/contrib/vaidya/src/java/org/apache/hadoop/vaidya/">hadoop
svn repository location</a>. 
+		   The default set of tests are under <code>"postexdiagnosis/tests/"</code>
folder.</p>
 		<ul>
 		  <li>Writing a test class for your new test case should extend the <code>org.apache.hadoop.vaidya.DiagnosticTest</code>
class and 
 		       it should override following three methods from the base class, 

Added: hadoop/mapreduce/trunk/src/docs/src/documentation/resources/images/mapreduce-logo.jpg
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/resources/images/mapreduce-logo.jpg?rev=816439&view=auto
==============================================================================
Binary file - no diff available.

Propchange: hadoop/mapreduce/trunk/src/docs/src/documentation/resources/images/mapreduce-logo.jpg
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Modified: hadoop/mapreduce/trunk/src/docs/src/documentation/skinconf.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/trunk/src/docs/src/documentation/skinconf.xml?rev=816439&r1=816438&r2=816439&view=diff
==============================================================================
--- hadoop/mapreduce/trunk/src/docs/src/documentation/skinconf.xml (original)
+++ hadoop/mapreduce/trunk/src/docs/src/documentation/skinconf.xml Fri Sep 18 02:20:48 2009
@@ -68,7 +68,7 @@
   <project-name>Hadoop</project-name>
   <project-description>Scalable Computing Platform</project-description>
   <project-url>http://hadoop.apache.org/core/</project-url>
-  <project-logo>images/core-logo.gif</project-logo>
+  <project-logo>images/mapreduce-logo.jpg</project-logo>
 
   <!-- group logo -->
   <group-name>Hadoop</group-name>
@@ -146,13 +146,13 @@
     <!--Headers -->
 	#content h1 {
 	  margin-bottom: .5em;
-	  font-size: 200%; color: black;
+	  font-size: 185%; color: black;
 	  font-family: arial;
 	}  
-    h2, .h3 { font-size: 195%; color: black; font-family: arial; }
-	h3, .h4 { font-size: 140%; color: black; font-family: arial; margin-bottom: 0.5em; }
+    h2, .h3 { font-size: 175%; color: black; font-family: arial; }
+	h3, .h4 { font-size: 135%; color: black; font-family: arial; margin-bottom: 0.5em; }
 	h4, .h5 { font-size: 125%; color: black;  font-style: italic; font-weight: bold; font-family:
arial; }
-	h5, h6 { font-size: 110%; color: #363636; font-weight: bold; } 
+	h5, h6 { font-size: 110%; color: #363636; font-weight: bold; }  
    
    <!--Code Background -->
     pre.code {



Mime
View raw message