Mailing-List: contact dev-help@felix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@felix.apache.org
MIME-Version: 1.0
In-Reply-To: <53FF3D89.6010307@bobpaulin.com>
References: <53FF27FE.1020401@bobpaulin.com>
	<CAKkCf4qxJmMx4L=-UZ--iwO2bY3Ty3XcXZkCWAASPZPTiOqvQg@mail.gmail.com>
	<53FF3D89.6010307@bobpaulin.com>
Date: Thu, 28 Aug 2014 16:53:47 +0200
Message-ID: 
 <CAKkCf4rWeVaod1KWPB=ncbTACeDtcXECgOm6Jrx7XPPP+tHAAg@mail.gmail.com>
Subject: Re: Event Admin Tuning the Async Thread Pool
From: Carsten Ziegeler <cziegeler@apache.org>
To: dev@felix.apache.org
Content-Type: multipart/alternative; boundary=001a113d586e3306680501b1b43f

--001a113d586e3306680501b1b43f
Content-Type: text/plain; charset=UTF-8

Hi Bob,

yes, the reason for the two pools is to actually avoid the deadlock
situation you mention.

Carsten


2014-08-28 16:32 GMT+02:00 Bob Paulin <bob@bobpaulin.com>:

> Carsten,
>
> I agree self-tuning single threadpool would be a very interesting
> implementation.  We should be able to adjust the maximum threads for a pool
> over a given interval.   I think making it a single thread pool would be
> the tricky part since I think we could deadlock if the entire active
> threadpool was filled with async threads.  But perhaps a timeout could be
> set to ensure that some of them get kicked out after a given amount of
> time.  I'll do some more research on that but in the mean time I'll submit
> a jira for the config option.
>
> - Bob
>
> On 8/28/2014 8:25 AM, Carsten Ziegeler wrote:
>
>> Hi Bob,
>>
>> this sounds good to me, however it would be even better if the event admin
>> could find out the ratio and adjust it accordingly. I'm not against making
>> this configurable, but it might be hard to know the good value upfront.
>>
>> As a general node, it would be great, if we could simply have a single
>> thread pool :)
>>
>> But +1 for a patch in any case.
>>
>> Carsten
>>
>>
>> 2014-08-28 15:00 GMT+02:00 Bob Paulin <bob@bobpaulin.com>:
>>
>>  Hi,
>>>
>>> I'd like to propose adding an additional configuration variable to Event
>>> Admin that controls the sync to async thread ratio.  Currently the code
>>> sets an intelligent default that sets the async threads at 50% of the
>>> sync
>>> threads as long as there are more than 5 total sync threads.
>>>
>>> final int asyncThreadPoolSize = m_threadPoolSize > 5 ? m_threadPoolSize /
>>> 2  : 2;
>>>
>>> This assumes an even split between post and send operations. However this
>>> is not the case with many applications leaning heavily one way or the
>>> other.  I'd like to replace the above code with:
>>>
>>> final int asyncThreadPoolSize = m_threadPoolSize > 5 ?
>>> Math.floor(m_threadPoolSize * m_asyncToSyncThreadRatio)  : 2;
>>>
>>> With m_asyncToSyncThreadRatio defaulting to 0.5.  This leaves the
>>> intelligent defaults but allows the admin to tune the application based
>>> on
>>> the application usage.  This tuning can show significant performance
>>> improvements.  From the EventAdmin StressTestIT:
>>>
>>> Scenerio 100% Async events
>>> Events sent 1050000
>>> Producer Threads 15
>>>
>>>
>>> @ 50% async to sync ratio
>>> PAXEXAM-PROBE-8762bbe7-91d9-4ff1-8562-38096af676da[org.
>>> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
>>> 1050000 events in 6551ms
>>>
>>> @ 100% async to sync ratio
>>> PAXEXAM-PROBE-02e70ff6-36ff-4bed-ab00-f2bcb9440109[org.
>>> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
>>> 1050000 events in 4871ms
>>>
>>> Thoughts?  Happy to submit a patch if there are no concerns.
>>>
>>> - Bob Paulin
>>>
>>>
>>
>>
>


-- 
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org

--001a113d586e3306680501b1b43f--