hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From d...@apache.org
Subject svn commit: r702360 [1/3] - in /hadoop/core/trunk: ./ docs/ src/docs/src/documentation/content/xdocs/ src/mapred/org/apache/hadoop/mapred/ src/test/org/apache/hadoop/mapred/ src/webapps/job/
Date Tue, 07 Oct 2008 06:55:30 GMT
Author: ddas
Date: Mon Oct  6 23:55:29 2008
New Revision: 702360

URL: http://svn.apache.org/viewvc?rev=702360&view=rev
Log:
HADOOP-4261. Adds a setup task for jobs. This is required so that we don't setup jobs that
haven't been inited yet (since init could lead to job failure). Only after the init has successfully
happened do we launch the setupJob task. Contributed by Amareshwari Sriramadasu.

Modified:
    hadoop/core/trunk/CHANGES.txt
    hadoop/core/trunk/docs/changes.html
    hadoop/core/trunk/docs/mapred_tutorial.html
    hadoop/core/trunk/docs/mapred_tutorial.pdf
    hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/EagerTaskInitializationListener.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/HistoryViewer.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobClient.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobHistory.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobInProgress.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobStatus.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobSubmissionProtocol.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobTracker.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/LocalJobRunner.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/MapTask.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/OutputCommitter.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/ReduceTask.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/RunningJob.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/Task.java
    hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/TaskInProgress.java
    hadoop/core/trunk/src/test/org/apache/hadoop/mapred/MiniMRCluster.java
    hadoop/core/trunk/src/test/org/apache/hadoop/mapred/TestJobTrackerRestart.java
    hadoop/core/trunk/src/test/org/apache/hadoop/mapred/TestMiniMRLocalFS.java
    hadoop/core/trunk/src/webapps/job/jobdetails.jsp
    hadoop/core/trunk/src/webapps/job/jobdetailshistory.jsp
    hadoop/core/trunk/src/webapps/job/jobtasks.jsp
    hadoop/core/trunk/src/webapps/job/taskdetails.jsp

Modified: hadoop/core/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/CHANGES.txt?rev=702360&r1=702359&r2=702360&view=diff
==============================================================================
--- hadoop/core/trunk/CHANGES.txt (original)
+++ hadoop/core/trunk/CHANGES.txt Mon Oct  6 23:55:29 2008
@@ -859,6 +859,11 @@
     HADOOP-4163. Report FSErrors from map output fetch threads instead of
     merely logging them. (Sharad Agarwal via cdouglas)
 
+    HADOOP-4261. Adds a setup task for jobs. This is required so that we 
+    don't setup jobs that haven't been inited yet (since init could lead
+    to job failure). Only after the init has successfully happened do we 
+    launch the setupJob task. (Amareshwari Sriramadasu via ddas)
+
 Release 0.18.2 - Unreleased
 
   BUG FIXES

Modified: hadoop/core/trunk/docs/changes.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/changes.html?rev=702360&r1=702359&r2=702360&view=diff
==============================================================================
--- hadoop/core/trunk/docs/changes.html (original)
+++ hadoop/core/trunk/docs/changes.html Mon Oct  6 23:55:29 2008
@@ -259,7 +259,7 @@
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._improvements_')">
 IMPROVEMENTS
-</a>&nbsp;&nbsp;&nbsp;(70)
+</a>&nbsp;&nbsp;&nbsp;(72)
     <ol id="release_0.19.0_-_unreleased_._improvements_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4205">HADOOP-4205</a>.
hive: metastore and ql to use the refactored SerDe library.<br />(zshao)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4106">HADOOP-4106</a>.
libhdfs: add time, permission and user attribute support
@@ -387,6 +387,8 @@
 make itermitant failures reproducible.<br />(szetszwo via cdouglas)</li>
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-4209">HADOOP-4209</a>.
Remove the change to the format of task attempt id by
 incrementing the task attempt numbers by 1000 when the job restarts.<br />(Amar Kamat
via omalley)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4301">HADOOP-4301</a>.
Adds forrest doc for the skip bad records feature.<br />(Sharad Agarwal via ddas)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4354">HADOOP-4354</a>.
Separate TestDatanodeDeath.testDatanodeDeath() into 4 tests.<br />(szetszwo)</li>
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._optimizations_')">
 OPTIMIZATIONS
@@ -413,7 +415,7 @@
     </ol>
   </li>
   <li><a href="javascript:toggleList('release_0.19.0_-_unreleased_._bug_fixes_')">
 BUG FIXES
-</a>&nbsp;&nbsp;&nbsp;(108)
+</a>&nbsp;&nbsp;&nbsp;(109)
     <ol id="release_0.19.0_-_unreleased_._bug_fixes_">
       <li><a href="http://issues.apache.org/jira/browse/HADOOP-3563">HADOOP-3563</a>.
 Refactor the distributed upgrade code so that it is
 easier to identify datanode and namenode related code.<br />(dhruba)</li>
@@ -621,6 +623,8 @@
 retries for fetching map-outputs; also fixed the case where the reducer
 automatically kills on too many unique map-outputs could not be fetched
 for small jobs.<br />(Amareshwari Sri Ramadasu via acmurthy)</li>
+      <li><a href="http://issues.apache.org/jira/browse/HADOOP-4163">HADOOP-4163</a>.
Report FSErrors from map output fetch threads instead of
+merely logging them.<br />(Sharad Agarwal via cdouglas)</li>
     </ol>
   </li>
 </ul>

Modified: hadoop/core/trunk/docs/mapred_tutorial.html
URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/mapred_tutorial.html?rev=702360&r1=702359&r2=702360&view=diff
==============================================================================
--- hadoop/core/trunk/docs/mapred_tutorial.html (original)
+++ hadoop/core/trunk/docs/mapred_tutorial.html Mon Oct  6 23:55:29 2008
@@ -2186,14 +2186,12 @@
 <li>
             Setup the job during initialization. For example, create
             the temporary output directory for the job during the
-            initialization of the job. The job client does the setup
-            for the job.
+            initialization of the job. 
           </li>
           
 <li>
             Cleanup the job after the job completion. For example, remove the
-            temporary output directory after the job completion. A separate 
-            task does the cleanupJob at the end of the job.
+            temporary output directory after the job completion.
           </li>
           
 <li>



Mime
View raw message