hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3992) Reduce fetcher doesn't verify HTTP status code of response
Date Fri, 09 Mar 2012 08:42:56 GMT
Reduce fetcher doesn't verify HTTP status code of response
----------------------------------------------------------

                 Key: MAPREDUCE-3992
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3992
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv1
    Affects Versions: 1.0.1
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon


Currently, the reduce fetch code doesn't check the HTTP status code of the response. This
can lead to the following situation:
- the map output servlet gets an IOException after setting the headers but before the first
call to flush()
- this causes it to send a response with a non-OK result code, including the exception text
as the response body (response.sendError() does this if the response isn't committed)
- it will still include the response headers indicating it's a valid response

In the case of a merge-to-memory, the compression codec might then try to interpret the HTML
response as compressed data, resulting in either a huge allocation (OOME) or some other nasty
error. This bug seems to be present in MR1, but haven't checked trunk/MR2 yet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message