Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Date: Mon, 14 Sep 2015 19:44:46 +0000 (UTC)
From: "Bob Hansen (JIRA)" <jira@apache.org>
To: hdfs-dev@hadoop.apache.org
Message-ID: <JIRA.12864000.1442259847000.333860.1442259886207@Atlassian.JIRA>
In-Reply-To: <JIRA.12864000.1442259847000@Atlassian.JIRA>
References: <JIRA.12864000.1442259847000@Atlassian.JIRA>
 <JIRA.12864000.1442259847419@arcas>
Subject: [jira] [Created] (HDFS-9078) WebHDFS does not support
 HTTP-keepalive
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Bob Hansen created HDFS-9078:
--------------------------------

             Summary: WebHDFS does not support HTTP-keepalive
                 Key: HDFS-9078
                 URL: https://issues.apache.org/jira/browse/HDFS-9078
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Bob Hansen


Since the inclusion of HDFS-7279, the netty back-end for WebHDFS is using HTTP 1.0, which does not support HDFS-keepalive.  In the same use case as HDFS-8855, where a WebHDFS client is seeking around a file, this creates a tremendous amount of connections to the datanode, leaving tens of thousands of connections in a TIME_WAIT state.  In some circumstances, all of the available ephemeral ports can be consumed with such connections and the datanode can fail.

Configuring netty to support HTTP 1.1 and HTTP-keepalive would allow clients to re-use a single connection (much like HDFS-8855) and keep the number of used ports constrained.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)