Return-Path: Delivered-To: apmail-hadoop-core-commits-archive@www.apache.org Received: (qmail 82714 invoked from network); 7 Oct 2008 06:55:53 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 7 Oct 2008 06:55:53 -0000 Received: (qmail 30307 invoked by uid 500); 7 Oct 2008 06:55:52 -0000 Delivered-To: apmail-hadoop-core-commits-archive@hadoop.apache.org Received: (qmail 30150 invoked by uid 500); 7 Oct 2008 06:55:52 -0000 Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-commits@hadoop.apache.org Received: (qmail 30140 invoked by uid 99); 7 Oct 2008 06:55:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Oct 2008 23:55:51 -0700 X-ASF-Spam-Status: No, hits=-1999.9 required=10.0 tests=ALL_TRUSTED,DNS_FROM_SECURITYSAGE X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Oct 2008 06:54:56 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 3066F238899B; Mon, 6 Oct 2008 23:55:32 -0700 (PDT) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r702360 [1/3] - in /hadoop/core/trunk: ./ docs/ src/docs/src/documentation/content/xdocs/ src/mapred/org/apache/hadoop/mapred/ src/test/org/apache/hadoop/mapred/ src/webapps/job/ Date: Tue, 07 Oct 2008 06:55:30 -0000 To: core-commits@hadoop.apache.org From: ddas@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20081007065532.3066F238899B@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: ddas Date: Mon Oct 6 23:55:29 2008 New Revision: 702360 URL: http://svn.apache.org/viewvc?rev=702360&view=rev Log: HADOOP-4261. Adds a setup task for jobs. This is required so that we don't setup jobs that haven't been inited yet (since init could lead to job failure). Only after the init has successfully happened do we launch the setupJob task. Contributed by Amareshwari Sriramadasu. Modified: hadoop/core/trunk/CHANGES.txt hadoop/core/trunk/docs/changes.html hadoop/core/trunk/docs/mapred_tutorial.html hadoop/core/trunk/docs/mapred_tutorial.pdf hadoop/core/trunk/src/docs/src/documentation/content/xdocs/mapred_tutorial.xml hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/EagerTaskInitializationListener.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/HistoryViewer.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobClient.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobHistory.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobInProgress.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobStatus.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobSubmissionProtocol.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobTracker.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/LocalJobRunner.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/MapTask.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/OutputCommitter.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/ReduceTask.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/RunningJob.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/Task.java hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/TaskInProgress.java hadoop/core/trunk/src/test/org/apache/hadoop/mapred/MiniMRCluster.java hadoop/core/trunk/src/test/org/apache/hadoop/mapred/TestJobTrackerRestart.java hadoop/core/trunk/src/test/org/apache/hadoop/mapred/TestMiniMRLocalFS.java hadoop/core/trunk/src/webapps/job/jobdetails.jsp hadoop/core/trunk/src/webapps/job/jobdetailshistory.jsp hadoop/core/trunk/src/webapps/job/jobtasks.jsp hadoop/core/trunk/src/webapps/job/taskdetails.jsp Modified: hadoop/core/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/core/trunk/CHANGES.txt?rev=702360&r1=702359&r2=702360&view=diff ============================================================================== --- hadoop/core/trunk/CHANGES.txt (original) +++ hadoop/core/trunk/CHANGES.txt Mon Oct 6 23:55:29 2008 @@ -859,6 +859,11 @@ HADOOP-4163. Report FSErrors from map output fetch threads instead of merely logging them. (Sharad Agarwal via cdouglas) + HADOOP-4261. Adds a setup task for jobs. This is required so that we + don't setup jobs that haven't been inited yet (since init could lead + to job failure). Only after the init has successfully happened do we + launch the setupJob task. (Amareshwari Sriramadasu via ddas) + Release 0.18.2 - Unreleased BUG FIXES Modified: hadoop/core/trunk/docs/changes.html URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/changes.html?rev=702360&r1=702359&r2=702360&view=diff ============================================================================== --- hadoop/core/trunk/docs/changes.html (original) +++ hadoop/core/trunk/docs/changes.html Mon Oct 6 23:55:29 2008 @@ -259,7 +259,7 @@
  • IMPROVEMENTS -   (70) +   (72)
    1. HADOOP-4205. hive: metastore and ql to use the refactored SerDe library.
      (zshao)
    2. HADOOP-4106. libhdfs: add time, permission and user attribute support @@ -387,6 +387,8 @@ make itermitant failures reproducible.
      (szetszwo via cdouglas)
    3. HADOOP-4209. Remove the change to the format of task attempt id by incrementing the task attempt numbers by 1000 when the job restarts.
      (Amar Kamat via omalley)
      4. +
    4. HADOOP-4301. Adds forrest doc for the skip bad records feature.
      (Sharad Agarwal via ddas)
      5. +
    5. HADOOP-4354. Separate TestDatanodeDeath.testDatanodeDeath() into 4 tests.
      (szetszwo)
  • OPTIMIZATIONS @@ -413,7 +415,7 @@
  • BUG FIXES -   (108) +   (109)
    1. HADOOP-3563. Refactor the distributed upgrade code so that it is easier to identify datanode and namenode related code.
      (dhruba)
      2. @@ -621,6 +623,8 @@ retries for fetching map-outputs; also fixed the case where the reducer automatically kills on too many unique map-outputs could not be fetched for small jobs.
      (Amareshwari Sri Ramadasu via acmurthy) +
    2. HADOOP-4163. Report FSErrors from map output fetch threads instead of +merely logging them.
      (Sharad Agarwal via cdouglas)
    • Modified: hadoop/core/trunk/docs/mapred_tutorial.html URL: http://svn.apache.org/viewvc/hadoop/core/trunk/docs/mapred_tutorial.html?rev=702360&r1=702359&r2=702360&view=diff ============================================================================== --- hadoop/core/trunk/docs/mapred_tutorial.html (original) +++ hadoop/core/trunk/docs/mapred_tutorial.html Mon Oct 6 23:55:29 2008 @@ -2186,14 +2186,12 @@
  • Setup the job during initialization. For example, create the temporary output directory for the job during the - initialization of the job. The job client does the setup - for the job. + initialization of the job.
  • Cleanup the job after the job completion. For example, remove the - temporary output directory after the job completion. A separate - task does the cleanupJob at the end of the job. + temporary output directory after the job completion.