oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cayanan, Michael D (388J)" <michael.d.caya...@jpl.nasa.gov>
Subject Workflow Questions
Date Tue, 02 Apr 2013 03:11:24 GMT
Hi all,

I'm trying to see if anyone knows of a cool, easy to use "Resource Manager" that can control
the amount of Workflows running concurrently on a node. Basically, my set up is like this:

- I have a Workflow Manager Server configured to run at most 10 Workflow Events concurrently.
- I have a bunch of files sitting on a RabbitMQ queue.
- I have a listener that is continually monitoring this particular queue. As soon as it sees
a file in this queue, it will trigger a Workflow event.

Under this scenario, normally if I have 30 files in the RabbitMQ, my listener will trigger
30 Workflow events. However, the Workflow Server would put 20 of these events in its repository
queue while the other 10 are running.

What I want some "Resource Manager" component to do is just run the 10 Workflow events only
and have the other 20 sitting in the RabbitMQ. As soon as 1 event has finished, then trigger
another workflow event to keep the number of concurrent workflows running at 10.
Eventually, we will want this "Resource Manager" component to be able to point to another
node, node 2, where it can decide to trigger another set of 10 Workflow events while node
1 is still running it's 10 Workflow events.

Hope this makes sense. Any helpful tips would be much appreciated.


View raw message