hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From d...@apache.org
Subject svn commit: r639662 [1/3] - in /hadoop/core/trunk: ./ docs/ src/contrib/index/src/java/org/apache/hadoop/contrib/index/main/ src/contrib/index/src/java/org/apache/hadoop/contrib/index/mapred/ src/docs/src/documentation/content/xdocs/ src/examples/org/a...
Date Fri, 21 Mar 2008 14:46:37 GMT
Author: ddas
Date: Fri Mar 21 07:46:01 2008
New Revision: 639662

URL: http://svn.apache.org/viewvc?rev=639662&view=rev
Log:
HADOOP-3041. Deprecates getOutputPath and defines two new APIs getCurrentOutputPath and getFinalOutputPath.
Contributed by Amareshwari Sriramadasu.

Modified:
    hadoop/core/trunk/CHANGES.txt
    hadoop/core/trunk/docs/changes.html
    hadoop/core/trunk/docs/mapred_tutorial.html
    hadoop/core/trunk/docs/mapred_tutorial.pdf
    hadoop/core/trunk/src/contrib/index/src/java/org/apache/hadoop/contrib/index/main/UpdateIndex.java
    hadoop/core/trunk/src/contrib/index/src/java/org/apache/hadoop/contrib/index/mapred/IndexUpdateOutputFormat.java
    hadoop/core/trunk/src/contrib/index/src/java/org/apache/hadoop/contrib/index/mapred/IndexUpdater.java
    hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
    hadoop/core/trunk/src/docs/src/documentation/content/xdocs/site.xml
    hadoop/core/trunk/src/examples/org/apache/hadoop/examples/RandomWriter.java
    hadoop/core/trunk/src/examples/org/apache/hadoop/examples/Sort.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobHistory.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/JobInProgress.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/LocalJobRunner.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/MapFileOutputFormat.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/OutputFormatBase.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/SequenceFileOutputFormat.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/Task.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/TaskTracker.java
    hadoop/core/trunk/src/java/org/apache/hadoop/mapred/TextOutputFormat.java
    hadoop/core/trunk/src/test/org/apache/hadoop/io/FileBench.java
    hadoop/core/trunk/src/test/org/apache/hadoop/mapred/GenericMRLoadGenerator.java
    hadoop/core/trunk/src/test/org/apache/hadoop/mapred/MRBench.java
    hadoop/core/trunk/src/test/org/apache/hadoop/mapred/SortValidator.java
    hadoop/core/trunk/src/test/org/apache/hadoop/mapred/ThreadedMapBenchmark.java

Modified: hadoop/core/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/CHANGES.txt?rev=639662&r1=639661&r2=639662&view=diff
==============================================================================
--- hadoop/core/trunk/CHANGES.txt (original)
+++ hadoop/core/trunk/CHANGES.txt Fri Mar 21 07:46:01 2008
@@ -351,6 +351,11 @@
     HADOOP-3042. Updates the Javadoc in JobConf.getOutputPath to reflect 
     the actual temporary path. (Amareshwari Sriramadasu via ddas)
 
+    HADOOP-3041. Deprecates getOutputPath and defines two new APIs
+    getCurrentOutputPath and getFinalOutputPath.
+    (Amareshwari Sriramadasu via ddas)
+
+
 Release 0.16.1 - 2008-03-13
 
   INCOMPATIBLE CHANGES

Modified: hadoop/core/trunk/docs/changes.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/changes.html?rev=639662&r1=639661&r2=639662&view=diff
==============================================================================
--- hadoop/core/trunk/docs/changes.html (original)
+++ hadoop/core/trunk/docs/changes.html Fri Mar 21 07:46:01 2008
@@ -102,7 +102,7 @@
     </ol>
   </li>
   <li><a href="javascript:toggleList('trunk_(unreleased_changes)_._improvements_')">
 IMPROVEMENTS
-</a>&nbsp;&nbsp;&nbsp;(22)
+</a>&nbsp;&nbsp;&nbsp;(23)
     <ol id="trunk_(unreleased_changes)_._improvements_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2655">HADOOP-2655</a>.
Copy on write for data and metadata files in the
 presence of snapshots. Needed for supporting appends to HDFS
@@ -146,6 +146,8 @@
 deallocation.<br />(Hemanth Yamijala via ddas)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2886">HADOOP-2886</a>.
 Track individual RPC metrics.<br />(girish vaitheeswaran via dhruba)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-2373">HADOOP-2373</a>.
Improvement in safe-mode reporting.<br />(shv)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-2796">HADOOP-2796</a>.
Enables distinguishing exit codes from user code vis-a-vis
+HOD's exit code.<br />(Hemanth Yamijala via ddas)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('trunk_(unreleased_changes)_._optimizations_')">
 OPTIMIZATIONS

Modified: hadoop/core/trunk/docs/mapred_tutorial.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/mapred_tutorial.html?rev=639662&r1=639661&r2=639662&view=diff
==============================================================================
--- hadoop/core/trunk/docs/mapred_tutorial.html (original)
+++ hadoop/core/trunk/docs/mapred_tutorial.html Fri Mar 21 07:46:01 2008
@@ -289,7 +289,7 @@
 <a href="#Example%3A+WordCount+v2.0">Example: WordCount v2.0</a>
 <ul class="minitoc">
 <li>
-<a href="#Source+Code-N10C11">Source Code</a>
+<a href="#Source+Code-N10C14">Source Code</a>
 </li>
 <li>
 <a href="#Sample+Runs">Sample Runs</a>
@@ -1798,11 +1798,15 @@
 <p>The application-writer can take advantage of this feature by 
           creating any side-files required in <span class="codefrag">${mapred.output.dir}</span>

           during execution of a task via 
-          <a href="api/org/apache/hadoop/mapred/JobConf.html#getOutputPath()">
-          JobConf.getOutputPath()</a>, and the framework will promote them 
+          <a href="api/org/apache/hadoop/mapred/JobConf.html#getCurrentOutputPath()">
+          JobConf.getCurrentOutputPath()</a>, and the framework will promote them 
           similarly for succesful task-attempts, thus eliminating the need to 
-          pick unique paths per task-attempt.</p>
-<a name="N10A84"></a><a name="RecordWriter"></a>
+          pick unique paths per task-attempt. she can get the actual configured 
+          path (final output path) via 
+          <a href="api/org/apache/hadoop/mapred/JobConf.html#getFinalOutputPath()">
+          JobConf.getFinalOutputPath()</a>
+</p>
+<a name="N10A87"></a><a name="RecordWriter"></a>
 <h4>RecordWriter</h4>
 <p>
 <a href="api/org/apache/hadoop/mapred/RecordWriter.html">
@@ -1810,9 +1814,9 @@
           pairs to an output file.</p>
 <p>RecordWriter implementations write the job outputs to the 
           <span class="codefrag">FileSystem</span>.</p>
-<a name="N10A9B"></a><a name="Other+Useful+Features"></a>
+<a name="N10A9E"></a><a name="Other+Useful+Features"></a>
 <h3 class="h4">Other Useful Features</h3>
-<a name="N10AA1"></a><a name="Counters"></a>
+<a name="N10AA4"></a><a name="Counters"></a>
 <h4>Counters</h4>
 <p>
 <span class="codefrag">Counters</span> represent global counters, defined either
by 
@@ -1826,7 +1830,7 @@
           Reporter.incrCounter(Enum, long)</a> in the <span class="codefrag">map</span>
and/or 
           <span class="codefrag">reduce</span> methods. These counters are then
globally 
           aggregated by the framework.</p>
-<a name="N10ACC"></a><a name="DistributedCache"></a>
+<a name="N10ACF"></a><a name="DistributedCache"></a>
 <h4>DistributedCache</h4>
 <p>
 <a href="api/org/apache/hadoop/filecache/DistributedCache.html">
@@ -1859,7 +1863,7 @@
           <a href="api/org/apache/hadoop/filecache/DistributedCache.html#createSymlink(org.apache.hadoop.conf.Configuration)">
           DistributedCache.createSymlink(Path, Configuration)</a> api. Files 
           have <em>execution permissions</em> set.</p>
-<a name="N10B0A"></a><a name="Tool"></a>
+<a name="N10B0D"></a><a name="Tool"></a>
 <h4>Tool</h4>
 <p>The <a href="api/org/apache/hadoop/util/Tool.html">Tool</a> 
           interface supports the handling of generic Hadoop command-line options.
@@ -1899,7 +1903,7 @@
             </span>
           
 </p>
-<a name="N10B3C"></a><a name="IsolationRunner"></a>
+<a name="N10B3F"></a><a name="IsolationRunner"></a>
 <h4>IsolationRunner</h4>
 <p>
 <a href="api/org/apache/hadoop/mapred/IsolationRunner.html">
@@ -1923,13 +1927,13 @@
 <p>
 <span class="codefrag">IsolationRunner</span> will run the failed task in a single

           jvm, which can be in the debugger, over precisely the same input.</p>
-<a name="N10B6F"></a><a name="JobControl"></a>
+<a name="N10B72"></a><a name="JobControl"></a>
 <h4>JobControl</h4>
 <p>
 <a href="api/org/apache/hadoop/mapred/jobcontrol/package-summary.html">
           JobControl</a> is a utility which encapsulates a set of Map-Reduce jobs
           and their dependencies.</p>
-<a name="N10B7C"></a><a name="Data+Compression"></a>
+<a name="N10B7F"></a><a name="Data+Compression"></a>
 <h4>Data Compression</h4>
 <p>Hadoop Map-Reduce provides facilities for the application-writer to
           specify compression for both intermediate map-outputs and the
@@ -1943,7 +1947,7 @@
           codecs for reasons of both performance (zlib) and non-availability of
           Java libraries (lzo). More details on their usage and availability are
           available <a href="native_libraries.html">here</a>.</p>
-<a name="N10B9C"></a><a name="Intermediate+Outputs"></a>
+<a name="N10B9F"></a><a name="Intermediate+Outputs"></a>
 <h5>Intermediate Outputs</h5>
 <p>Applications can control compression of intermediate map-outputs
             via the 
@@ -1964,7 +1968,7 @@
             <a href="api/org/apache/hadoop/mapred/JobConf.html#setMapOutputCompressionType(org.apache.hadoop.io.SequenceFile.CompressionType)">
             JobConf.setMapOutputCompressionType(SequenceFile.CompressionType)</a> 
             api.</p>
-<a name="N10BC8"></a><a name="Job+Outputs"></a>
+<a name="N10BCB"></a><a name="Job+Outputs"></a>
 <h5>Job Outputs</h5>
 <p>Applications can control compression of job-outputs via the
             <a href="api/org/apache/hadoop/mapred/OutputFormatBase.html#setCompressOutput(org.apache.hadoop.mapred.JobConf,%20boolean)">
@@ -1984,7 +1988,7 @@
 </div>
 
     
-<a name="N10BF7"></a><a name="Example%3A+WordCount+v2.0"></a>
+<a name="N10BFA"></a><a name="Example%3A+WordCount+v2.0"></a>
 <h2 class="h3">Example: WordCount v2.0</h2>
 <div class="section">
 <p>Here is a more complete <span class="codefrag">WordCount</span> which
uses many of the
@@ -1994,7 +1998,7 @@
       <a href="quickstart.html#SingleNodeSetup">pseudo-distributed</a> or
       <a href="quickstart.html#Fully-Distributed+Operation">fully-distributed</a>

       Hadoop installation.</p>
-<a name="N10C11"></a><a name="Source+Code-N10C11"></a>
+<a name="N10C14"></a><a name="Source+Code-N10C14"></a>
 <h3 class="h4">Source Code</h3>
 <table class="ForrestTable" cellspacing="1" cellpadding="4">
           
@@ -3204,7 +3208,7 @@
 </tr>
         
 </table>
-<a name="N11373"></a><a name="Sample+Runs"></a>
+<a name="N11376"></a><a name="Sample+Runs"></a>
 <h3 class="h4">Sample Runs</h3>
 <p>Sample text-files as input:</p>
 <p>
@@ -3372,7 +3376,7 @@
 <br>
         
 </p>
-<a name="N11447"></a><a name="Highlights"></a>
+<a name="N1144A"></a><a name="Highlights"></a>
 <h3 class="h4">Highlights</h3>
 <p>The second version of <span class="codefrag">WordCount</span> improves
upon the 
         previous one by using some features offered by the Map-Reduce framework:



Mime
View raw message