Return-Path: X-Original-To: apmail-hadoop-mapreduce-commits-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1101ECF5F for ; Tue, 10 Jul 2012 14:51:36 +0000 (UTC) Received: (qmail 82018 invoked by uid 500); 10 Jul 2012 14:51:35 -0000 Delivered-To: apmail-hadoop-mapreduce-commits-archive@hadoop.apache.org Received: (qmail 81766 invoked by uid 500); 10 Jul 2012 14:51:32 -0000 Mailing-List: contact mapreduce-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-commits@hadoop.apache.org Received: (qmail 81677 invoked by uid 99); 10 Jul 2012 14:51:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Jul 2012 14:51:31 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Jul 2012 14:51:28 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id E70492388847; Tue, 10 Jul 2012 14:51:06 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1359708 - in /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project: CHANGES.txt hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java Date: Tue, 10 Jul 2012 14:51:06 -0000 To: mapreduce-commits@hadoop.apache.org From: bobby@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20120710145106.E70492388847@eris.apache.org> Author: bobby Date: Tue Jul 10 14:51:06 2012 New Revision: 1359708 URL: http://svn.apache.org/viewvc?rev=1359708&view=rev Log: svn merge -c 1302754 FIXES: MAPREDUCE-3992. Reduce fetcher doesn't verify HTTP status code of response. Contributed by Todd Lipcon. Modified: hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java Modified: hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt?rev=1359708&r1=1359707&r2=1359708&view=diff ============================================================================== --- hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt (original) +++ hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt Tue Jul 10 14:51:06 2012 @@ -304,6 +304,9 @@ Release 0.23.3 - UNRELEASED MAPREDUCE-4300. OOM in AM can turn it into a zombie. (Robert Evans via tgraves) + MAPREDUCE-3992. Reduce fetcher doesn't verify HTTP status code of response + (todd) + Release 0.23.2 - UNRELEASED INCOMPATIBLE CHANGES Modified: hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java?rev=1359708&r1=1359707&r2=1359708&view=diff ============================================================================== --- hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java (original) +++ hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java Tue Jul 10 14:51:06 2012 @@ -23,6 +23,7 @@ import java.io.InputStream; import java.io.OutputStream; import java.net.MalformedURLException; import java.net.URL; +import java.net.HttpURLConnection; import java.net.URLConnection; import java.util.HashSet; import java.util.List; @@ -204,7 +205,7 @@ class Fetcher extends Thread { try { URL url = getMapOutputURL(host, maps); - URLConnection connection = url.openConnection(); + HttpURLConnection connection = (HttpURLConnection)url.openConnection(); // generate hash of the url String msgToEncode = SecureShuffleUtils.buildMsgFrom(url); @@ -218,6 +219,14 @@ class Fetcher extends Thread { connect(connection, connectionTimeout); connectSucceeded = true; input = new DataInputStream(connection.getInputStream()); + + // Validate response code + int rc = connection.getResponseCode(); + if (rc != HttpURLConnection.HTTP_OK) { + throw new IOException( + "Got invalid response code " + rc + " from " + url + + ": " + connection.getResponseMessage()); + } // get the replyHash which is HMac of the encHash we sent to the server String replyHash = connection.getHeaderField(SecureShuffleUtils.HTTP_HEADER_REPLY_URL_HASH);