hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Denny Ye (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2384) Can MR make error response Immediately?
Date Tue, 15 Mar 2011 04:42:29 GMT
Can MR make error response Immediately?
---------------------------------------

                 Key: MAPREDUCE-2384
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2384
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: job submission
    Affects Versions: 0.21.0
            Reporter: Denny Ye


When I read the source code of MapReduce in Hadoop 0.21.0, sometimes it made me confused about
error response. For example:
        1. JobSubmitter checking output for each job. MapReduce makes rule to limit that each
job output must be not exist to avoid fault overwrite. In my opinion, MR should verify output
at the point of client submitting. Actually, it copies related files to specified target and
then, doing the verifying. 
        2. JobTracker.   Job has been submitted to JobTracker. In first step, JT create JIT
object that is very "huge" . Next step, JT start to verify job queue authority and memory
requirements.
 
        In normal case, verifying client input then response immediately if any cases in fault.
Regular logic can be performed if all the inputs have passed.  
        It seems like that those code does not make sense for understanding. Is only my personal
opinion? Wish someone help me to explain the details. Thanks!

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message