ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srimanth Gunturi <sgunt...@hortonworks.com>
Subject Re: ThreadPoolExecutor being used incorrectly
Date Thu, 14 Jan 2016 18:40:19 GMT
Hi Jayush,
It is very nice that you have noticed the same issue as well and have a solution - I will
look into it.

I have a working patch myself where I use a custom rejection handler that relies on a secondary
queue (unbounded), and made the Executor's queue bounded (like Michael Harp mentioned).

I will give both a spin and write down in JIRA.
Regards,
Srimanth



________________________________________
From: Jayush Luniya <jluniya@hortonworks.com>
Sent: Thursday, January 14, 2016 10:32 AM
To: dev@ambari.apache.org
Subject: Re: ThreadPoolExecutor being used incorrectly

Hi Srimanth,

I recently noticed this behavior too. I filed an external JIRA with a
prototype to address this issue.
https://issues.apache.org/jira/browse/AMBARI-14671


Regards
Jayush

On 1/14/16, 10:01 AM, "Srimanth Gunturi" <sgunturi@hortonworks.com> wrote:

>Hello,
>Wanted to write this down and have discussion about
>'java.util.concurrent.ThreadPoolExecutor' being used incorrectly in
>Ambari.
>
>Almost all usages call the below constructor:
>
>public ThreadPoolExecutor(
>                              int corePoolSize,
>                              int maximumPoolSize,
>                              long keepAliveTime,
>                              TimeUnit unit,
>                              BlockingQueue<Runnable> workQueue)
>
>Where typical values used are:
>
>new ThreadPoolExecutor(
>            20,
>            100,
>            30000L,
>            TimeUnit.MILLISECONDS,
>            new LinkedBlockingQueue<Runnable>()) // Unbounded queue
>
>The weird thing about 'ThreadPoolExecutor' is that the number of threads
>increases from the 'corePoolSize' (20) to the 'maximumPoolSize' (100)
>ONLY when the BlockingQueue is full. Whenever we use an unbounded
>blocking queue, the number of threads never goes past the 'corePoolSize'.
>
>The javadoc states: "If there are more than corePoolSize but less than
>maximumPoolSize threads running, a new thread will be created only if the
>queue is full"
>
>So our thinking that the Executor will automatically increase the number
>of threads upto 'maximumPoolSize' and store overflowing requests into the
>queue (unbounded) is incorrect. Due to our usage, the number of threads
>never increases beyond 'corePoolSize'.
>
>I do not understand the reason why it is implemented this way, but we run
>into performance issues by getting stuck with 'corePoolSize' thread
>count. I am looking into a fix for using ThreadPoolExecutor where the
>number of threads increases upto 'maximumPoolSize'.
>
>Regards,
>Srimanth?
>



Mime
View raw message