river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: A new implementation of TaskManager
Date Wed, 07 Jul 2010 14:41:54 GMT
Patricia Shanahan wrote:
> Peter Firmstone wrote:
>> So it would seem that the runAfter() method is superfluous and 
>> unnecessary in most, if not all cases?  Especially if the caller is 
>> making the decision about when tasks are placed on the queue.
>> We do need to confirm that in our current implementations, the 
>> callers are taking responsibility for the queue placement.  Then try 
>> to discovery why we needed to have a dependency check by the 
>> TaskManager.
>> It sounds like for absolute concurrency the design needs to be 
>> totally revised, perhaps the responsibility of determining if the 
>> Task is ready to be executed should be placed solely on the Task 
>> implementation itself, but not by passing in the list of all Tasks on 
>> the queue, but with a simple:
> Do you have a lot of bug reports indicating the concurrency design is 
> basically broken?
No, not really, don't think it's full of bugs, just not as concurrent as 
it should be, I'd like to see it plateau under pressure, rather than 
degrade significantly.   I'm also a bit puzzled why the caller, 
TaskManager and Task all share some responsibility (but not all 
responsibility) for managing dependency ordering correctness.

> In a few months, when I have had time to learn my way around both the 
> specification and the implementation, I would be happy to participate 
> in a liveness and ordering review. I've never done it for software, 
> but some of the intellectual tools used for proving memory order 
> properties in large multiprocessor designs were borrowed from 
> distributed software engineering.
Sounds like you've got some good experiences to share.

> I am opposed to moving the queuing and run after organization work 
> from TaskManager to its callers. 
I understand, if based on the work involved for each implementing 
class.  To reduce the work substantially, a class implementing Task 
(eg:: TaskHelper) could be provided for the developer implementing a 
Task to encapsulate, to delegate the dependency management to 
internally, rather than extending an abstract class.  This reduces the 
work back to mostly implementing one method, run(), but leaves the 
implementer free to do something else too.  

It does have some merit:

    * Very small lists and references are fast, with small lock windows
      and good concurrency.
    * The caller doesn't talk to TaskManager, only Task, coupling is
    * The caller doesn't have to be responsible for the safety of
      placing Tasks on a queue.
    * Any number of initiators can add dependencies to a Task.
    * TaskManager can assume all Tasks received are ready to run.
    * The Task implementer can encapsulate a TaskHelper, to delegate
      responsibility, for dependencies and correctness. 

> It is work that has to be done, and the choice is between doing it in 
> one class, or doing it in each class that needs to both permit some 
> parallelism and yet maintain some ordering among tasks. It is far less 
> work to make it both efficient and correct if it only needs to be done 
> in one class.
But I digress, it's only a suggestion and it might not be the optimum 
solution, so feel free to do something else.



View raw message