hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhihai xu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.
Date Mon, 23 Feb 2015 08:47:12 GMT
zhihai xu created MAPREDUCE-6265:
------------------------------------

             Summary: make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better
control the thread pool size to launch/kill containers.
                 Key: MAPREDUCE-6265
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: mrv2
            Reporter: zhihai xu
            Assignee: zhihai xu


make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread
pool size to launch/kill containers
Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
{code}
  protected static final int INITIAL_POOL_SIZE = 10;
{code}
We should make it configurable because the thread pool size will be decided by INITIAL_POOL_SIZE,
limitOnPoolSize and number of node used by the AM.
Since we already made limitOnPoolSize configurable, it make senses to also make INITIAL_POOL_SIZE
configurable to better manage the thread pool size.
We saw some issue due to the small thread pool size when some node is down. The recovery from
a shutdown node take very long time due to all the ContainerLauncher threads are blocked by
IPC client connection to the shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message