Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Date: Mon, 3 Feb 2014 11:16:11 +0000 (UTC)
From: "Harsh J (JIRA)" <jira@apache.org>
To: hdfs-dev@hadoop.apache.org
Message-ID: <JIRA.12400457.1216288233090.26532.1391426171648@arcas>
In-Reply-To: <JIRA.12400457.1216288233090@arcas>
References: <JIRA.12400457.1216288233090@arcas>
Subject: [jira] [Resolved] (HDFS-285) limit concurrent connections(data
 serving thread) in one datanode
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Harsh J resolved HDFS-285.
--------------------------

    Resolution: Not A Problem

This has likely gone stale (probably addressed at a higher level via Raghu's earliest comments).

In having seen some pretty large HBase region sets on several clusters, and never having faced the described stack limit OOME (but having faced the transceiver limits) I think this is likely no longer an issue.

Closing out as 'Not a Problem' (anymore).

> limit concurrent connections(data serving thread) in one datanode
> -----------------------------------------------------------------
>
>                 Key: HDFS-285
>                 URL: https://issues.apache.org/jira/browse/HDFS-285
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Luo Ning
>            Priority: Minor
>
> i'm here after HADOOP-2341 and HADOOP-2346, in my hbase env, many opening mapfiles cause datanode OOME(stack memory), because 2000+ data serving threads in datanode process.
> although HADOOP-2346 has implements timeouts, it will be some situation many connection created  before the read timeout(default 6min) reach. like hbase does, it open all files on regionserver startup. 
> limit concurrent connections(data serving thread) will make datanode more stable. and i think it could be done in SocketIOWithTimeout$SelectorPool#select:
> 1. in SelectorPool#select, record all waiting SelectorInfo instances in a List at the beginning, and remove it after 'Selector#select' done.
> 2. before real 'select',  do a limitation check, if reached, close the first selectorInfo. 


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)