hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6967) DNs may OOM under high webhdfs load
Date Thu, 28 Aug 2014 19:16:10 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daryn Sharp updated HDFS-6967:
------------------------------

    Assignee: Eric Payne

> DNs may OOM under high webhdfs load
> -----------------------------------
>
>                 Key: HDFS-6967
>                 URL: https://issues.apache.org/jira/browse/HDFS-6967
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode, webhdfs
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Assignee: Eric Payne
>
> Webhdfs uses jetty.  The size of the request thread pool is limited, but jetty will accept
and queue infinite connections.  Every queued connection is "heavy" with buffers, etc.  Unlike
data streamer connections, thousands of webhdfs connections will quickly OOM a DN.  The accepted
requests must be bounded and excess clients rejected so they retry on a new DN.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message