hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-1773) ShuffleHeader should have a format that can inform about errors
Date Sat, 01 Mar 2014 01:19:19 GMT
Bikas Saha created YARN-1773:
--------------------------------

             Summary: ShuffleHeader should have a format that can inform about errors
                 Key: YARN-1773
                 URL: https://issues.apache.org/jira/browse/YARN-1773
             Project: Hadoop YARN
          Issue Type: Bug
    Affects Versions: 2.3.0
            Reporter: Bikas Saha


Currently, the ShuffleHeader (which is a Writable) simply tries to read the successful header
(mapid, reduceid etc). If there is an error then the input will have an error message instead
of (mapid, reducedid etc). Thus parsing the ShuffleHeader fails and since we dont know where
the error message ends, we cannot consume the remaining input stream which may have good data
from the remaining map outputs. Being able to encode the error in the ShuffleHeader will let
us parse out the error correctly and move on to the remaining data.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message