oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Workflow Questions
Date Tue, 02 Apr 2013 05:15:02 GMT
Hey Mike,

I'll bite, can't you use the Resource Manager and its queues/nodes to do


Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA

-----Original Message-----
From: <Cayanan>, "Michael D   (388J)" <michael.d.cayanan@jpl.nasa.gov>
Reply-To: "user@oodt.apache.org" <user@oodt.apache.org>
Date: Monday, April 1, 2013 8:11 PM
To: "user@oodt.apache.org" <user@oodt.apache.org>
Subject: Workflow Questions

>Hi all,
>I'm trying to see if anyone knows of a cool, easy to use "Resource
>Manager" that can control the amount of Workflows running concurrently on
>a node. Basically, my set up is like this:
>- I have a Workflow Manager Server configured to run at most 10 Workflow
>Events concurrently.
>- I have a bunch of files sitting on a RabbitMQ queue.
>- I have a listener that is continually monitoring this particular queue.
>As soon as it sees a file in this queue, it will trigger a Workflow event.
>Under this scenario, normally if I have 30 files in the RabbitMQ, my
>listener will trigger 30 Workflow events. However, the Workflow Server
>would put 20 of these events in its repository queue while the other 10
>are running.
>What I want some "Resource Manager" component to do is just run the 10
>Workflow events only and have the other 20 sitting in the RabbitMQ. As
>soon as 1 event has finished, then trigger another workflow event to keep
>the number of concurrent workflows running
> at 10.
>Eventually, we will want this "Resource Manager" component to be able to
>point to another node, node 2, where it can decide to trigger another set
>of 10 Workflow events while node 1 is still running it's 10 Workflow
>Hope this makes sense. Any helpful tips would be much appreciated.

View raw message