hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Hansen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9078) WebHDFS does not support HTTP-keepalive
Date Mon, 14 Sep 2015 19:44:46 GMT
Bob Hansen created HDFS-9078:
--------------------------------

             Summary: WebHDFS does not support HTTP-keepalive
                 Key: HDFS-9078
                 URL: https://issues.apache.org/jira/browse/HDFS-9078
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Bob Hansen


Since the inclusion of HDFS-7279, the netty back-end for WebHDFS is using HTTP 1.0, which
does not support HDFS-keepalive.  In the same use case as HDFS-8855, where a WebHDFS client
is seeking around a file, this creates a tremendous amount of connections to the datanode,
leaving tens of thousands of connections in a TIME_WAIT state.  In some circumstances, all
of the available ephemeral ports can be consumed with such connections and the datanode can
fail.

Configuring netty to support HTTP 1.1 and HTTP-keepalive would allow clients to re-use a single
connection (much like HDFS-8855) and keep the number of used ports constrained.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message