commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Steitz <phil.ste...@gmail.com>
Subject Re: [math] Multithreaded performances
Date Wed, 06 Nov 2013 18:08:02 GMT
On 11/6/13 9:05 AM, Romain Manni-Bucau wrote:
> Great!
>
> Btw not sure for sirona we oculd use it. One constraint on sirona-core
> is to stay self contained. We already shade math3 so shading pool2 too
> would start to create a big jar for this need. I'll try to bench
> deeper next week too.

OK - and any ideas you have about how to implement something
lightweight inside [math] much appreciated.

Phil
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
>
>
>
> 2013/11/6 Phil Steitz <phil.steitz@gmail.com>:
>> On 11/6/13 8:47 AM, Romain Manni-Bucau wrote:
>>> well pool are based on locks so I'm not sure (it would need deep
>>> benchs on a real app) it does worth it
>> Commons pool2 uses pretty lightweight locking and using a pool of
>> instances achieves the basic objective of reducing contention for
>> the single sync lock on one SummaryStatistics object.   I bet it
>> would improve throughput over the single-instance approach if
>> maxActive, maxIdle were tuned.  If I get some time to play with
>> this, I will report back with some benchmarks.
>>
>> Phil
>>> Romain Manni-Bucau
>>> Twitter: @rmannibucau
>>> Blog: http://rmannibucau.wordpress.com/
>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>> Github: https://github.com/rmannibucau
>>>
>>>
>>>
>>> 2013/11/6 Phil Steitz <phil.steitz@gmail.com>:
>>>> On 11/5/13 11:26 PM, Romain Manni-Bucau wrote:
>>>>> Hehe, right.
>>>>>
>>>>> I looked a bit more today and LongAdder is only a part of the
>>>>> solution. The stat computation still needs to lock to get acces to
>>>>> previous values (N -> N+1). Basically the gain wouldn't be as
>>>>> important as I thought :(.
>>>> Right, but I think your original idea of maintaining a pool of
>>>> instances (fewer that one per thread) to be periodically aggregated
>>>> is a good one.  See below.
>>>>> As I said before we'll wait a bit to gather feedbacks, if it blocks
>>>>> I'll come back trying to find + propose a solution.
>>>>>
>>>>> Thanks in all cases for your answers!
>>>> A workaround that I have started playing with (partly for other
>>>> benchmarking reasons) might be to actually use a pool for the stats
>>>> objects that the monitoring threads use.  Using a pool would allow
>>>> you to monitor and tune the parameters.  We now have (well, once the
>>>> VOTE in progress completes :) a decently performing pool
>>>> implementation.  The tricky bit is locking the instances during
>>>> aggregation.  One way to handle this would be to have the factory's
>>>> passivate method and the aggregation thread contend for locks on the
>>>> pooled stats instances.  The only contention would be when
>>>> aggregation is copying individual instances and contention would be
>>>> with at most one client thread (waiting to proceed in passivate).
>>>>
>>>> Phil
>>>>> Romain Manni-Bucau
>>>>> Twitter: @rmannibucau
>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>> Github: https://github.com/rmannibucau
>>>>>
>>>>>
>>>>>
>>>>> 2013/11/5 Phil Steitz <phil.steitz@gmail.com>:
>>>>>> On 11/5/13 9:57 AM, Romain Manni-Bucau wrote:
>>>>>>> @Phil: hmm can be but the framework would create its own overhead
which
>>>>>>> would be avoided with a dedicated solution, no? Well thought
gain was great
>>>>>>> for small investment but ok to postpone it
>>>>>> As I said, patches welcome.  Go for it.  My point about the
>>>>>> framework was that when you actually get this implemented inside,
>>>>>> e.g. SummaryStatistics,  you will have built a mini-framework.
>>>>>> Whatever overhead it has, it will have ;)
>>>>>>
>>>>>>
>>>>>> Phil
>>>>>>
>>>>>>
>>>>>>> Le 5 nov. 2013 18:54, "Romain Manni-Bucau" <rmannibucau@gmail.com>
a écrit :
>>>>>>>
>>>>>>>> Well I didnt test sirona in prod but when using jamon (same
kind of
>>>>>>>> framework) locks were creating a serious overhead on some
benches. Not the
>>>>>>>> most important but enough to try to solve it.
>>>>>>>>
>>>>>>>> That said we are not yet in 1.0 so Im ok to wait for more
serious
>>>>>>>> feedbacks if you think it is better
>>>>>>>> Le 5 nov. 2013 18:48, "Ted Dunning" <ted.dunning@gmail.com>
a écrit :
>>>>>>>>
>>>>>>>>> On Mon, Nov 4, 2013 at 10:09 PM, Romain Manni-Bucau
>>>>>>>>> <rmannibucau@gmail.com>wrote:
>>>>>>>>>
>>>>>>>>>> Oh sorry, that's what I said early, in a real app
no or not enough to
>>>>>>>>> be an
>>>>>>>>>> issue buy on simple apps or very high thrououtput
apps yes.
>>>>>>>>>>  Le 5 nov. 2013 07:00, "Ted Dunning" <ted.dunning@gmail.com>
a écrit :
>>>>>>>>>>
>>>>>>>>>>> That isn't what I meant.
>>>>>>>>>>>
>>>>>>>>>>> Do you really think that more than one metric
has to update
>>>>>>>>> (increment,
>>>>>>>>>>> say) at precisely the same time?
>>>>>>>>>>>
>>>>>>>>> I realize that is what you said.  Do you have any serious
examples where
>>>>>>>>> metrics have to be updated all or nothing?
>>>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message