Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Sat, 21 May 2011 18:42:47 +0000 (UTC)
From: "Harsh J Chouraria (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <1059184853.33683.1306003367410.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1632625961.2842.1300164149550.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (MAPREDUCE-2384) Can MR make error response
 Immediately?
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Harsh J Chouraria updated MAPREDUCE-2384:
-----------------------------------------

    Fix Version/s: 0.23.0
     Release Note: Submitter should fail on errors early, before transferring files.
           Status: Patch Available  (was: Open)

As before, I do not think refactoring (2) is a good idea maintenance-wise. Here's a patch for just the reordering of (1). Some simple jobsubs pass with the change -- I believe existing test cases cover this change already; but let me know if not.

> Can MR make error response Immediately?
> ---------------------------------------
>
>                 Key: MAPREDUCE-2384
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2384
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: job submission
>    Affects Versions: 0.21.0
>            Reporter: Denny Ye
>            Assignee: Harsh J Chouraria
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2384.r1.diff
>
>
> When I read the source code of MapReduce in Hadoop 0.21.0, sometimes it made me confused about error response. For example:
>         1. JobSubmitter checking output for each job. MapReduce makes rule to limit that each job output must be not exist to avoid fault overwrite. In my opinion, MR should verify output at the point of client submitting. Actually, it copies related files to specified target and then, doing the verifying. 
>         2. JobTracker.   Job has been submitted to JobTracker. In first step, JT create JIT object that is very "huge" . Next step, JT start to verify job queue authority and memory requirements.
>  
>         In normal case, verifying client input then response immediately if any cases in fault. Regular logic can be performed if all the inputs have passed.  
>         It seems like that those code does not make sense for understanding. Is only my personal opinion? Wish someone help me to explain the details. Thanks!

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira