river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gregg Wonderly <gr...@wonderly.org>
Subject Re: TaskManager
Date Fri, 01 Jul 2011 02:22:57 GMT
On 6/30/2011 9:02 PM, Patricia Shanahan wrote:
> I'm having trouble squaring this with the description of
> ThreadPoolExecutor. If I'm reading it right, it *can* be initialized to
> set a low limit on the maximum number of threads, but can equally well
> be set to have up to Integer.MAX_VALUE threads, effectively unlimited.

Yes, this is the case.  There are some other subtle behaviors which we just need 
to study to make sure that the "documented behavior" is what we need.

> There are two forms of TaskManager behavior that I do not see an easy
> way to reproduce using a simple ThreadPoolExecutor, runAfter and loadLevel.
>
> runAfter my be better handled by the class issuing the tasks.

There is also the Fork-Join framework which does allow some "arrangement" of 
order.  I'm not sure this is a great fit right now, but it is something to 
consider I think, especially for "notifications", "commits" and some other 
things that have two level ordering "function->tasks".

> loadLevel permits specification of a ratio between threads and tasks, so
> that threads are only created up to e.g. one third of the number of
> tasks. I'm not sure that is a really good idea. It means leaving tasks
> to wait even if we could create another thread.

I don't like this thought at all in a distributed system.  A vast majority of 
tasks really need to be invoked immediately to maintain a reasonable chance at 
throughput.

Gregg

> Patricia
>
>
>
> On 6/29/2011 7:06 AM, Christopher Dolan wrote:
>> Just remember, we discussed this a year ago
>> (http://mail-archives.apache.org/mod_mbox/incubator-river-dev/201006.mbo
>> x/%3C4C12147A.8030601@zeus.net.au%3E) and some of us concluded that
>> ThreadPoolExecutor has rather non-intuitive runtime behavior because
>> it's optimized to not cause OutOfMemory problems. Not everyone had a
>> problem with that behavior, but I'm not a fan.
>>
>> Chris
>>
>> P.S. can someone add a link to the Incubator mail archives to the web
>> page? They are quite hard to find...
>> http://mail-archives.apache.org/mod_mbox/incubator-river-dev/
>> http://mail-archives.apache.org/mod_mbox/incubator-river-user/
>> http://mail-archives.apache.org/mod_mbox/incubator-river-commits/
>>
>>
>> -----Original Message-----
>> From: Peter Firmstone [mailto:jini@zeus.net.au]
>> Sent: Sunday, June 26, 2011 5:18 PM
>> To: dev@river.apache.org
>> Subject: Re: TaskManager
>>
>>
>>> I think the way to look at this is to examine the non-trivial runAfter
>>> implementations. If they have common elements that can be helped by
>> some
>>> utility classes, write those classes. If not, deal with each
>>> individually.
>>>
>>> We are now committed to 1.5, so if we did not have runAfter we could
>>> replace TaskManager with java.util.concurrent.ThreadPoolExecutor.
>>
>> That's probably where we should be headed, since it reduces our
>> maintenance burden.
>>
>> Peter.
>>
>
>


Mime
View raw message