hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denny Ye <denny...@gmail.com>
Subject Can MR make error response Immediately?
Date Fri, 11 Mar 2011 04:10:10 GMT
hi gays,

       When I read the source code of MapReduce in Hadoop 0.21.0, sometimes
it made me confused about error response. For example:
        1. JobSubmitter checking output for each job. MapReduce makes rule
to limit that each job output must be not exist to avoid fault overwrite. In
my opinion, MR should verify output at the point of client submitting.
Actually, it copies related files to specified target and then, doing the
verifying.
        2. JobTracker.   Job has been submitted to JobTracker. In first
step, JT create JIT object that is very "huge" . Next step, JT start to
verify job queue authority and memory requirements.

        In normal case, verifying client input then response immediately if
any cases in fault. Regular logic can be performed if all the inputs have
passed.
        It seems like that those code does not make sense for understanding.
Is only my personal opinion? Wish someone help me to explain the details.
Thanks!



Regards
-- Denny Ye

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message