river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: Smoothed PooledThreadExecutor
Date Thu, 11 Aug 2011 10:58:37 GMT
I like this approach, it sounds elegant.

Cheers,

Peter.

Patricia Shanahan wrote:
> I've thought of a significant simplification, based on the idea that I
> only need samples of the task count.
>
> I was going to try to use a concurrent queue and deal with observations
> getting out of order. I just realized I can make it much cleaner and
> simpler by using a ReentrantLock. The manager thread would use
> lockInterruptibly, and wait indefinitely to get the lock if another
> thread holds it.
>
> A thread that has just made a change to the task count would use
> tryLock(). If it gets the lock, it can add its report to a simple
> LinkedList. If it fails to get the lock it would just carry on without
> collecting a sample. Either the management thread is processing
> collected data, or another thread is logging the task count at about the
> same time. Either way it does not need to collect a sample.
>
> This strategy automatically throttles the number of observations that
> get collected for the manager to consider, though I may also want a way
> to further limit the number of messages. However, that can be done more
> simply, and with better information, inside code controlled by the lock.
>
> Patricia
>
>
> On 8/9/2011 12:36 PM, Patricia Shanahan wrote:
>> I'm investigating extended ThreadPoolExecutor to get an Executor that
>> bases the number of threads on the mean number of tasks over an
>> interval. If successful, that would flatten out sudden peaks in the task
>> load, using the queue as a buffer, but would make the number of threads
>> match a sustained load, so that there would be no unnecessary 
>> bottleneck.
>>
>> The current TaskManager allows specification of e.g. one thread per
>> three tasks, which does result in a lower rate of change but can leave
>> the number of threads a fraction of the number of tasks in a sustained
>> workload.
>>
>> Here is a general outline of a design:
>>
>> Methods that change the number of tasks would log the new number of
>> tasks and a timestamp on a concurrent queue. As a refinement, there may
>> be an option to use a ThreadLocal Random to log only with a specified
>> probability to bound the number of messages on the queue.
>>
>> A thread (more about which thread later) would periodically drain the
>> queue into a local TreeSet, getting them sorted, and then calculate the
>> mean number of tasks and adjust the thread count accordingly.
>>
>> The big design problem is which thread does this work. Here are the
>> options I'm considering:
>>
>> 1. Add it to the tasks, so that a ThreadPoolExecutor worker thread does
>> it. The downside is that all the existing worker threads could get busy
>> on long running tasks and tasks are building up in the queue. Nothing
>> would be done about it until a worker thread finishes its current task.
>>
>> 2. Do it from the execute method, in whatever thread is adding the task.
>> Again, we could get into situations in which nothing is done about a
>> situation that could be improved by adding threads. The effect would be
>> to make the execute method randomly slow from the point of view of its
>> caller.
>>
>> 3. Use a dedicated thread that sleeps until it is time to reconsider the
>> number of tasks, processes the queue of log messages, does any
>> adjustment, and then goes back to sleep. This is the most reliable in
>> terms of getting adjustments made when they are needed, but costs the
>> overhead of an extra thread that is just managing the thread pool.
>>
>> Any views, alternatives?
>>
>> Thanks,
>>
>> Patricia
>>
>
>


Mime
View raw message