commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Steitz <phil.ste...@gmail.com>
Subject Re: [math] Multithreaded performances
Date Wed, 06 Nov 2013 17:02:49 GMT
On 11/6/13 8:47 AM, Romain Manni-Bucau wrote:
> well pool are based on locks so I'm not sure (it would need deep
> benchs on a real app) it does worth it

Commons pool2 uses pretty lightweight locking and using a pool of
instances achieves the basic objective of reducing contention for
the single sync lock on one SummaryStatistics object.   I bet it
would improve throughput over the single-instance approach if
maxActive, maxIdle were tuned.  If I get some time to play with
this, I will report back with some benchmarks.

Phil
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
>
>
>
> 2013/11/6 Phil Steitz <phil.steitz@gmail.com>:
>> On 11/5/13 11:26 PM, Romain Manni-Bucau wrote:
>>> Hehe, right.
>>>
>>> I looked a bit more today and LongAdder is only a part of the
>>> solution. The stat computation still needs to lock to get acces to
>>> previous values (N -> N+1). Basically the gain wouldn't be as
>>> important as I thought :(.
>> Right, but I think your original idea of maintaining a pool of
>> instances (fewer that one per thread) to be periodically aggregated
>> is a good one.  See below.
>>> As I said before we'll wait a bit to gather feedbacks, if it blocks
>>> I'll come back trying to find + propose a solution.
>>>
>>> Thanks in all cases for your answers!
>> A workaround that I have started playing with (partly for other
>> benchmarking reasons) might be to actually use a pool for the stats
>> objects that the monitoring threads use.  Using a pool would allow
>> you to monitor and tune the parameters.  We now have (well, once the
>> VOTE in progress completes :) a decently performing pool
>> implementation.  The tricky bit is locking the instances during
>> aggregation.  One way to handle this would be to have the factory's
>> passivate method and the aggregation thread contend for locks on the
>> pooled stats instances.  The only contention would be when
>> aggregation is copying individual instances and contention would be
>> with at most one client thread (waiting to proceed in passivate).
>>
>> Phil
>>> Romain Manni-Bucau
>>> Twitter: @rmannibucau
>>> Blog: http://rmannibucau.wordpress.com/
>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>> Github: https://github.com/rmannibucau
>>>
>>>
>>>
>>> 2013/11/5 Phil Steitz <phil.steitz@gmail.com>:
>>>> On 11/5/13 9:57 AM, Romain Manni-Bucau wrote:
>>>>> @Phil: hmm can be but the framework would create its own overhead which
>>>>> would be avoided with a dedicated solution, no? Well thought gain was
great
>>>>> for small investment but ok to postpone it
>>>> As I said, patches welcome.  Go for it.  My point about the
>>>> framework was that when you actually get this implemented inside,
>>>> e.g. SummaryStatistics,  you will have built a mini-framework.
>>>> Whatever overhead it has, it will have ;)
>>>>
>>>>
>>>> Phil
>>>>
>>>>
>>>>> Le 5 nov. 2013 18:54, "Romain Manni-Bucau" <rmannibucau@gmail.com>
a écrit :
>>>>>
>>>>>> Well I didnt test sirona in prod but when using jamon (same kind
of
>>>>>> framework) locks were creating a serious overhead on some benches.
Not the
>>>>>> most important but enough to try to solve it.
>>>>>>
>>>>>> That said we are not yet in 1.0 so Im ok to wait for more serious
>>>>>> feedbacks if you think it is better
>>>>>> Le 5 nov. 2013 18:48, "Ted Dunning" <ted.dunning@gmail.com>
a écrit :
>>>>>>
>>>>>>> On Mon, Nov 4, 2013 at 10:09 PM, Romain Manni-Bucau
>>>>>>> <rmannibucau@gmail.com>wrote:
>>>>>>>
>>>>>>>> Oh sorry, that's what I said early, in a real app no or not
enough to
>>>>>>> be an
>>>>>>>> issue buy on simple apps or very high thrououtput apps yes.
>>>>>>>>  Le 5 nov. 2013 07:00, "Ted Dunning" <ted.dunning@gmail.com>
a écrit :
>>>>>>>>
>>>>>>>>> That isn't what I meant.
>>>>>>>>>
>>>>>>>>> Do you really think that more than one metric has to
update
>>>>>>> (increment,
>>>>>>>>> say) at precisely the same time?
>>>>>>>>>
>>>>>>> I realize that is what you said.  Do you have any serious examples
where
>>>>>>> metrics have to be updated all or nothing?
>>>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message