ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jayush Luniya (JIRA)" <>
Subject [jira] [Updated] (AMBARI-14671) Reevaluate the use of threadpools in Ambari code base
Date Thu, 02 Feb 2017 06:05:51 GMT


Jayush Luniya updated AMBARI-14671:
    Fix Version/s:     (was: 2.5.0)

> Reevaluate the use of threadpools in Ambari code base
> -----------------------------------------------------
>                 Key: AMBARI-14671
>                 URL:
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.2.0
>            Reporter: Jayush Luniya
>            Assignee: Jayush Luniya
>             Fix For: 3.0.0
>         Attachments: DynamicScaling.tgz
> As part of investigation for BUG-43981, noticed that in many places in the Ambari code
base, the way we use the threadpool is not quite correct.  We will never scale up the number
of threads on high load when we use unbounded queues. This could lead to performance bottlenecks
especially if we are configure ThreadPoolExecutor with corePoolSize=0 and maxPoolSize=10,
only one thread will ever be spawned. See observations below
> Observations: 
> 1. When a ThreadPoolExecutor object is created, the pool size is 0 (i.e. no new threads
are created then) unless prestartAllCoreThreads() is called. Also if we set allowCoreThreadTimeOut(true),
idle core threads will also be reclaimed.
> 2. In our code base, I observed that we create a threadpool using unlimited queue. However
the pool will never scale up from coreThreads -> maxThreads as the request will always
get queued. 
> {code:java}
> LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>();
// unlimited Queue
> ThreadPoolExecutor threadPoolExecutor =
>     new ThreadPoolExecutor(
>         TimeUnit.MILLISECONDS,
>         queue);
> {code}
> {quote}
> Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without
a predefined capacity) will cause new tasks to wait in the queue when all corePoolSize threads
are busy. Thus, no more than corePoolSize threads will ever be created. (And the value of
the maximumPoolSize therefore doesn't have any effect.) This may be appropriate when each
task is completely independent of others, so tasks cannot affect each others execution; for
example, in a web page server. While this style of queuing can be useful in smoothing out
transient bursts of requests, it admits the possibility of unbounded work queue growth when
commands continue to arrive on average faster than they can be processed.
> {quote}

This message was sent by Atlassian JIRA

View raw message