ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srimanth Gunturi <sgunt...@hortonworks.com>
Subject ThreadPoolExecutor being used incorrectly
Date Thu, 14 Jan 2016 18:01:04 GMT
Hello,
Wanted to write this down and have discussion about 'java.util.concurrent.ThreadPoolExecutor'
being used incorrectly in Ambari.

Almost all usages call the below constructor:

public ThreadPoolExecutor(
                              int corePoolSize,
                              int maximumPoolSize,
                              long keepAliveTime,
                              TimeUnit unit,
                              BlockingQueue<Runnable> workQueue)

Where typical values used are:

new ThreadPoolExecutor(
            20,
            100,
            30000L,
            TimeUnit.MILLISECONDS,
            new LinkedBlockingQueue<Runnable>()) // Unbounded queue

The weird thing about 'ThreadPoolExecutor' is that the number of threads increases from the
'corePoolSize' (20) to the 'maximumPoolSize' (100) ONLY when the BlockingQueue is full. Whenever
we use an unbounded blocking queue, the number of threads never goes past the 'corePoolSize'.

The javadoc states: "If there are more than corePoolSize but less than maximumPoolSize threads
running, a new thread will be created only if the queue is full"

So our thinking that the Executor will automatically increase the number of threads upto 'maximumPoolSize'
and store overflowing requests into the queue (unbounded) is incorrect. Due to our usage,
the number of threads never increases beyond 'corePoolSize'.

I do not understand the reason why it is implemented this way, but we run into performance
issues by getting stuck with 'corePoolSize' thread count. I am looking into a fix for using
ThreadPoolExecutor where the number of threads increases upto 'maximumPoolSize'.

Regards,
Srimanth?


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message