hadoop-mapreduce-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tomwh...@apache.org
Subject svn commit: r951492 - in /hadoop/mapreduce/branches/branch-0.21: ./ src/docs/src/documentation/content/xdocs/
Date Fri, 04 Jun 2010 16:58:39 GMT
Author: tomwhite
Date: Fri Jun  4 16:58:38 2010
New Revision: 951492

URL: http://svn.apache.org/viewvc?rev=951492&view=rev
Log:
Merge -r 951485:951486 from trunk to branch-0.21. Fixes: MAPREDUCE-1404

Removed:
    hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/cluster_setup.xml
    hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/commands_manual.xml
    hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
Modified:
    hadoop/mapreduce/branches/branch-0.21/CHANGES.txt
    hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/capacity_scheduler.xml
    hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
    hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/site.xml

Modified: hadoop/mapreduce/branches/branch-0.21/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/branch-0.21/CHANGES.txt?rev=951492&r1=951491&r2=951492&view=diff
==============================================================================
--- hadoop/mapreduce/branches/branch-0.21/CHANGES.txt (original)
+++ hadoop/mapreduce/branches/branch-0.21/CHANGES.txt Fri Jun  4 16:58:38 2010
@@ -739,6 +739,9 @@ Release 0.21.0 - Unreleased
 
     MAPREDUCE-1832. Allow file sizes less than 1MB in DFSIO benchmark. (shv)
 
+    MAPREDUCE-1404.  Move Cluster-Setup and Single-Node-Setup Docs from
+    MapReduce to Common.  (tomwhite)
+
   OPTIMIZATIONS
 
     MAPREDUCE-270. Fix the tasktracker to optionally send an out-of-band

Modified: hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/capacity_scheduler.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/capacity_scheduler.xml?rev=951492&r1=951491&r2=951492&view=diff
==============================================================================
--- hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/capacity_scheduler.xml
(original)
+++ hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/capacity_scheduler.xml
Fri Jun  4 16:58:38 2010
@@ -87,7 +87,7 @@
         <li>
           Support for refreshing/reloading some of the queue-properties
           without restarting the JobTracker, taking advantage of the
-          <a href="cluster_setup.html#Refreshing+queue+configuration">
+          <a href="ext:cluster-setup/RefreshingQueueConfiguration">
           queue-refresh</a> feature in the framework.
         </li>
       </ul>
@@ -160,7 +160,7 @@
           You can define multiple, possibly hierarchical queues to which users
           can submit jobs with the Capacity Scheduler. To define queues,
           various properties should be set in two configuration files -
-          <a href="cluster_setup.html#mapred-queues.xml">mapred-queues.xml</a>
+          <a href="ext:cluster-setup/mapred-queues.xml">mapred-queues.xml</a>
           and
           <a href="ext:capacity-scheduler-conf">conf/capacity-scheduler.xml</a>
           .</p>
@@ -194,7 +194,7 @@
         <p>To specify a property for a specific queue that is defined in the
         mapred-queues.xml, you should set the corresponding property in a
         &lt;property&gt; tag explained
-        <a href="cluster_setup.html#property_tag">here</a>.
+        <a href="ext:cluster-setup/property_tag">here</a>.
         </p>
 
         <p>The properties defined for queues and their descriptions are
@@ -204,7 +204,7 @@
           <tr>
           	<th>Name</th>
             <th>
-            	<a href="commands_manual.html#RefreshQueues">
+            	<a href="ext:commands-manual/RefreshQueues">
             	Refresh-able?</a>
            	</th>
             <th>Applicable to?</th>

Modified: hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml?rev=951492&r1=951491&r2=951492&view=diff
==============================================================================
--- hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
(original)
+++ hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
Fri Jun  4 16:58:38 2010
@@ -44,7 +44,7 @@
           <a href="ext:single-node-setup">Single Node Setup</a> for first-time
users.
         </li>
         <li>
-          <a href="cluster_setup.html">Cluster Setup</a> for large, distributed
clusters.
+          <a href="ext:cluster-setup">Cluster Setup</a> for large, distributed
clusters.
         </li>
       </ul>
     </section>
@@ -159,7 +159,7 @@
       
       <p>This example works with a 
       pseudo-distributed (<a href="ext:single-node-setup">Single Node Setup</a>)

-     or fully-distributed (<a href="cluster_setup.html">Cluster Setup</a>) 
+     or fully-distributed (<a href="ext:cluster-setup">Cluster Setup</a>) 
       Hadoop installation.</p>   
       
       <section>
@@ -392,7 +392,7 @@
         provides this functionality is called the <em>distributed cache</em>.
         More details about the command line options surrounding job launching
         and control of the distributed cache are available at 
-        <a href="commands_manual.html"> Hadoop Commands Guide.</a></p>
+        <a href="ext:commands-manual"> Hadoop Commands Guide.</a></p>
         
         <p>Hadoop ships with some example code in a jar precompiled for you;
         one of these is (another) wordcount program. Here's an example
@@ -1214,7 +1214,7 @@
         <p>Note: <code>mapred.{map|reduce}.child.java.opts</code> are used
only 
         for configuring the launched child tasks from task tracker. Configuring 
         the memory options for daemons is documented under
-        <a href="cluster_setup.html#Configuring+the+Environment+of+the+Hadoop+Daemons">
+        <a href="ext:cluster-setup/ConfiguringEnvironmentHadoopDaemons">
         Configuring the Environment of the Hadoop Daemons</a> (Cluster Setup).</p>
         
         <p>The memory available to some parts of the framework is also
@@ -1626,6 +1626,60 @@
             </li>
           </ul>
         </section>
+        
+        <section>
+          <title>Job Authorization</title>
+          <p>Job level authorization is enabled on the cluster, if the configuration
+          <code>mapreduce.cluster.job-authorization-enabled</code> is set to
+          true. When enabled, access control checks are done by the JobTracker
+          and the TaskTracker before allowing users to view
+          job details or to modify a job using Map/Reduce APIs,
+          CLI or web user interfaces.</p>
+         
+          <p>A job submitter can specify access control lists for viewing or
+          modifying a job via the configuration properties
+          <code>mapreduce.job.acl-view-job</code> and
+          <code>mapreduce.job.acl-modify-job</code> respectively. By default,

+          nobody is given access in these properties.</p> 
+          
+          <p>However, irrespective of the ACLs configured, a job's owner,
+          the superuser and the members of an admin configured supergroup
+          (<code>mapreduce.cluster.permissions.supergroup</code>) always
+          have access to view and modify a job.</p>
+          
+          <p> A job view ACL authorizes users against the configured 
+          <code>mapreduce.job.acl-view-job</code> before returning possibly 
+          sensitive information about a job, like: </p>
+          <ul>
+            <li> job level counters </li>
+            <li> task level counters </li>
+            <li> tasks's diagnostic information </li>
+            <li> task logs displayed on the TaskTracker web UI </li>
+            <li> job.xml showed by the JobTracker's web UI </li>
+          </ul>
+          <p>Other information about a job, like its status and its profile, 
+          is accessible to all users, without requiring authorization.</p>
+          
+          <p> A job modification ACL authorizes users against the configured
+          <code>mapreduce.job.acl-modify-job</code> before allowing
+          modifications to jobs, like: </p>
+          <ul>
+            <li> killing a job </li>
+            <li> killing/failing a task of a job </li>
+            <li> setting the priority of a job </li>
+          </ul>
+          <p>These operations are also protected by the queue level ACL,
+          "acl-administer-jobs", configured via mapred-queue-acls.xml. The caller
+          will be authorized against both queue level ACLs and job level ACLs,
+          depending on what is enabled.</p>
+          
+          <p>The format of a job level ACL is the same as the format for a
+          queue level ACL as defined in the
+          <a href ="ext:cluster-setup/ConfiguringHadoopDaemons">
+          Cluster Setup</a> documentation.
+          </p>
+          
+        </section>
       </section>
 
       <section>
@@ -2366,7 +2420,7 @@
       <p>This example needs the HDFS to be up and running, especially for the 
       <code>DistributedCache</code>-related features. Hence it only works with
a 
       pseudo-distributed (<a href="ext:single-node-setup">Single Node Setup</a>)

-     or fully-distributed (<a href="cluster_setup.html#Fully-Distributed+Operation">Cluster
Setup</a>) 
+     or fully-distributed (<a href="ext:cluster-setup/FullyDistributedOperation">Cluster
Setup</a>) 
       Hadoop installation.</p>     
       
       <section>

Modified: hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/site.xml
URL: http://svn.apache.org/viewvc/hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/site.xml?rev=951492&r1=951491&r2=951492&view=diff
==============================================================================
--- hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/site.xml
(original)
+++ hadoop/mapreduce/branches/branch-0.21/src/docs/src/documentation/content/xdocs/site.xml
Fri Jun  4 16:58:38 2010
@@ -34,13 +34,11 @@ See http://forrest.apache.org/docs/linki
   
    <docs label="Getting Started"> 
 		<overview   				label="Overview" 					href="index.html" />
-		<setup label="Cluster Setup" href="cluster_setup.html"/>
 		<mapred    				label="MapReduce Tutorial" 	href="mapred_tutorial.html" />
 		 <streaming 				label="Hadoop Streaming"  href="streaming.html" />
    </docs>	
 		
   <docs label="Guides">
-		<commands 				label="Hadoop Commands"  href="commands_manual.html" />
 		<distcp    					label="DistCp"       href="distcp.html" />
 		<vaidya    					label="Vaidya" 		href="vaidya.html"/>
 		<archives  				label="Hadoop Archives"     href="hadoop_archives.html"/>
@@ -50,7 +48,6 @@ See http://forrest.apache.org/docs/linki
     <docs label="Schedulers">
         <cap_scheduler 		label="Capacity Scheduler"     href="capacity_scheduler.html"/>
 		<fair_scheduler 			label="Fair Scheduler"            href="fair_scheduler.html"/>
-		<cap_scheduler 		label="Hod Scheduler" 			href="hod_scheduler.html"/>
     </docs>
    
    <docs label="Miscellaneous"> 
@@ -83,6 +80,17 @@ See http://forrest.apache.org/docs/linki
       <PreReqs href="#PreReqs" />
       <Download href="#Download" />
     </single-node-setup>
+    <cluster-setup href="http://hadoop.apache.org/common/docs/current/cluster_setup.html">
+      <property_tag href="#property_tag" />
+      <RefreshingQueueConfiguration href="#Refreshing+queue+configuration"/>
+      <mapred-queues.xml href="#mapred-queues.xml"/>
+      <ConfiguringEnvironmentHadoopDaemons href="#Configuring+the+Environment+of+the+Hadoop+Daemons"/>
+      <ConfiguringHadoopDaemons href="#Configuring+the+Hadoop+Daemons"/>
+      <FullyDistributedOperation href="#Fully-Distributed+Operation"/>
+    </cluster-setup>
+    <commands-manual href="http://hadoop.apache.org/common/docs/current/commands_default.html">
+      <RefreshQueues href="#RefreshQueues"/>
+    </commands-manual>
 
     <zlib      href="http://www.zlib.net/" />
     <gzip      href="http://www.gzip.org/" />



Mime
View raw message