hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s..@apache.org
Subject svn commit: r1064918 - in /hadoop/common/branches/branch-0.22: CHANGES.txt src/docs/src/documentation/content/xdocs/cluster_setup.xml src/docs/src/documentation/content/xdocs/hod_scheduler.xml
Date Fri, 28 Jan 2011 22:42:37 GMT
Author: shv
Date: Fri Jan 28 22:42:37 2011
New Revision: 1064918

URL: http://svn.apache.org/viewvc?rev=1064918&view=rev
Log:
HADOOP-6812. Merge -r 1064916:1064917 from trunk to branch 0.22.

Modified:
    hadoop/common/branches/branch-0.22/CHANGES.txt
    hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
    hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml

Modified: hadoop/common/branches/branch-0.22/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/CHANGES.txt?rev=1064918&r1=1064917&r2=1064918&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/CHANGES.txt (original)
+++ hadoop/common/branches/branch-0.22/CHANGES.txt Fri Jan 28 22:42:37 2011
@@ -209,6 +209,11 @@ Release 0.22.0 - Unreleased
 
     HADOOP-7110. Implement chmod with JNI. (todd)
 
+    HADOOP-6812. Change documentation for correct placement of configuration
+    variables: mapreduce.reduce.input.buffer.percent, 
+    mapreduce.task.io.sort.factor, mapreduce.task.io.sort.mb
+    (Chris Douglas via shv)
+
   OPTIMIZATIONS
 
     HADOOP-6884. Add LOG.isDebugEnabled() guard for each LOG.debug(..).

Modified: hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml?rev=1064918&r1=1064917&r2=1064918&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
(original)
+++ hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/cluster_setup.xml
Fri Jan 28 22:42:37 2011
@@ -666,22 +666,33 @@
                     </td>
                   </tr>
                   <tr>
-                    <td>conf/core-site.xml</td>
-                    <td>fs.inmemory.size.mb</td>
-                    <td>200</td>
+                    <td>conf/mapred-site.xml</td>
+                    <td>mapreduce.reduce.shuffle.input.buffer.percent</td>
+                    <td>0.80</td>
                     <td>
-                      Larger amount of memory allocated for the in-memory 
-                      file-system used to merge map-outputs at the reduces.
+                      Larger amount of memory allocated for merging map output
+                      in memory during the shuffle. Expressed as a fraction of
+                      the total heap.
                     </td>
                   </tr>
                   <tr>
-                    <td>conf/core-site.xml</td>
+                    <td>conf/mapred-site.xml</td>
+                    <td>mapreduce.reduce.input.buffer.percent</td>
+                    <td>0.80</td>
+                    <td>
+                      Larger amount of memory allocated for retaining map output
+                      in memory during the reduce. Expressed as a fraction of
+                      the total heap.
+                    </td>
+                  </tr>
+                  <tr>
+                    <td>conf/mapred-site.xml</td>
                     <td>mapreduce.task.io.sort.factor</td>
                     <td>100</td>
                     <td>More streams merged at once while sorting files.</td>
                   </tr>
                   <tr>
-                    <td>conf/core-site.xml</td>
+                    <td>conf/mapred-site.xml</td>
                     <td>mapreduce.task.io.sort.mb</td>
                     <td>200</td>
                     <td>Higher memory-limit while sorting data.</td>

Modified: hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml?rev=1064918&r1=1064917&r2=1064918&view=diff
==============================================================================
--- hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
(original)
+++ hadoop/common/branches/branch-0.22/src/docs/src/documentation/content/xdocs/hod_scheduler.xml
Fri Jan 28 22:42:37 2011
@@ -250,7 +250,7 @@ info_port = Port number of the HDFS Name
   <p>For example:</p>
 <source>
 server-params = mapred.reduce.parallel.copies=20,io.sort.factor=100,io.sort.mb=128,io.file.buffer.size=131072
-final-server-params = mapred.child.java.opts=-Xmx512m,dfs.block.size=134217728,fs.inmemory.size.mb=128
  
+final-server-params = mapred.child.java.opts=-Xmx512m,dfs.block.size=134217728
 </source>
   <p>In order to provide the options from command line, you can use the following syntax:</p>
   <p>For configuring the MapReduce daemons use:</p>



Mime
View raw message