Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Fri, 9 Mar 2012 21:16:59 +0000 (UTC)
From: "Steve Loughran (Commented) (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <374240749.45581.1331327819510.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1881365339.42733.1331282575710.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-3992) Reduce fetcher doesn't verify
 HTTP status code of response
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226470#comment-13226470 ] 

Steve Loughran commented on MAPREDUCE-3992:
-------------------------------------------

@Todd, which class should I look at here?

the content-length header is what should really be set, as that allows the recipient to recognise another failure mode.

# the output servlet successfully writes all the data
# it calls flush() to flush the buffer to the socket stream.
# the connection goes down through the miracle that is TCP/IP
# the recipient sees the end of the HTTP response, and assumes that the entire dataset has arrived safely.
# The error propagates.

There are also risks of corruption in the payload that neither the Ether or TCP protocols pick up on; every layer has a flaw.

See
 * Loughran 2011, _did you really want that data_ [Presentation|http://www.slideshare.net/steve_l/did-you-reallywantthatdata]
 * Stone & Partridge 2000, _When The CRC and TCP Checksum Disagree_ [Paper|http://www-static.cc.gatech.edu/classes/AY2002/cs8803d_spring/papers/checksum.pdf]

Packet corruption is more likely long-haul than in-datacentre, so I'm not going to advocate the Content-MD5 overhead, but content-length can only be a good thing, provide the response length is known in advance (which it never is with JSP pages, but that's a separate issue)
                
> Reduce fetcher doesn't verify HTTP status code of response
> ----------------------------------------------------------
>
>                 Key: MAPREDUCE-3992
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3992
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv1
>    Affects Versions: 1.0.1
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>
> Currently, the reduce fetch code doesn't check the HTTP status code of the response. This can lead to the following situation:
> - the map output servlet gets an IOException after setting the headers but before the first call to flush()
> - this causes it to send a response with a non-OK result code, including the exception text as the response body (response.sendError() does this if the response isn't committed)
> - it will still include the response headers indicating it's a valid response
> In the case of a merge-to-memory, the compression codec might then try to interpret the HTML response as compressed data, resulting in either a huge allocation (OOME) or some other nasty error. This bug seems to be present in MR1, but haven't checked trunk/MR2 yet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira