ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jayush Luniya (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-14671) Reevaluate the use of threadpools in Ambari code base
Date Thu, 14 Jan 2016 18:25:40 GMT
Jayush Luniya created AMBARI-14671:
--------------------------------------

             Summary: Reevaluate the use of threadpools in Ambari code base
                 Key: AMBARI-14671
                 URL: https://issues.apache.org/jira/browse/AMBARI-14671
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.2.0
            Reporter: Jayush Luniya
            Assignee: Jayush Luniya
             Fix For: 2.4.0


As part of investigation for BUG-43981, noticed that in many places in the Ambari code base,
the way we use the threadpool is not quite correct.  We will never scale up the number of
threads on high load when we use unbounded queues. This could lead to performance bottlenecks
especially if we are configure ThreadPoolExecutor with corePoolSize=0 and maxPoolSize=10,
only one thread will ever be spawned. See observations below

Observations: 
1. When a ThreadPoolExecutor object is created, the pool size is 0 (i.e. no new threads are
created then) unless prestartAllCoreThreads() is called. Also if we set allowCoreThreadTimeOut(true),
idle core threads will also be reclaimed.
2. In our code base, I observed that we create a threadpool using unlimited queue. However
the pool will never scale up from coreThreads -> maxThreads as the request will always
get queued. 

{code:java}
LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue<Runnable>(); //
unlimited Queue
ThreadPoolExecutor threadPoolExecutor =
    new ThreadPoolExecutor(
        THREAD_POOL_CORE_SIZE,
        THREAD_POOL_MAX_SIZE,
        THREAD_POOL_TIMEOUT_MILLIS,
        TimeUnit.MILLISECONDS,
        queue);
{code}
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
{quote}
Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue without a predefined
capacity) will cause new tasks to wait in the queue when all corePoolSize threads are busy.
Thus, no more than corePoolSize threads will ever be created. (And the value of the maximumPoolSize
therefore doesn't have any effect.) This may be appropriate when each task is completely independent
of others, so tasks cannot affect each others execution; for example, in a web page server.
While this style of queuing can be useful in smoothing out transient bursts of requests, it
admits the possibility of unbounded work queue growth when commands continue to arrive on
average faster than they can be processed.
{quote}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message