Mailing-List: contact user-help@oodt.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@oodt.apache.org
Received-SPF: pass (athena.apache.org: local policy)
From: "Cayanan, Michael D (388J)" <michael.d.cayanan@jpl.nasa.gov>
To: "user@oodt.apache.org" <user@oodt.apache.org>
Subject: Re: Workflow Questions
Thread-Topic: Workflow Questions
Thread-Index: AQHOL0/sOzPHOyrG30mZ79CnA5ZY8ZjC2McAgAA3CgA=
Date: Tue, 2 Apr 2013 15:32:03 +0000
Message-ID: <CD803D9D.189C0%michael.cayanan@jpl.nasa.gov>
In-Reply-To: <CD7FB752.C4DAA%chris.a.mattmann@jpl.nasa.gov>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.3.2.130206
Content-Type: text/plain; charset="us-ascii"
Content-ID: <C36C26138A30E948A10244FE627596C3@ad.jpl>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hey Chris,

I suppose I could. For some reason I didn't think that it was possible to
submit a job to the Resource Manager other than through the Workflow.
In my case, I'm using a listener to pull files out of the RabbitMQ. After
I pull a file out of the queue, I'd like to submit a job to the Resource
Manager to trigger a Workflow Event. How would one submit a job this way?
Under the Resource Manager trunk, in src/main/resources/examples/jobs, I
see 2 xml files that appear to be config files. Do I need to make use of
these to do what I want to do?

Also, I know that the Resource Manager has it's own queue where it places
jobs when all of its compute nodes are full. In my case (this is for
Costin's pipeline BTW), I don't want jobs being placed in the Resource
Manager queue. I would like for them to remain in the Rabbit Queue. Is the
Resource Manager's "getNodeLoad()" method the best way to check if a node
is full before submitting a job?

Thanks in advance for your help!

-Mike


On 4/1/13 10:15 PM, "Mattmann, Chris A (388J)"
<chris.a.mattmann@jpl.nasa.gov> wrote:

>Hey Mike,
>
>I'll bite, can't you use the Resource Manager and its queues/nodes to do
>this?
>
>Cheers,
>Chris
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Senior Computer Scientist
>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>Office: 171-266B, Mailstop: 171-246
>Email: chris.a.mattmann@nasa.gov
>WWW:  http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Adjunct Assistant Professor, Computer Science Department
>University of Southern California, Los Angeles, CA 90089 USA
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
>-----Original Message-----
>From: <Cayanan>, "Michael D   (388J)" <michael.d.cayanan@jpl.nasa.gov>
>Reply-To: "user@oodt.apache.org" <user@oodt.apache.org>
>Date: Monday, April 1, 2013 8:11 PM
>To: "user@oodt.apache.org" <user@oodt.apache.org>
>Subject: Workflow Questions
>
>>Hi all,
>>
>>
>>I'm trying to see if anyone knows of a cool, easy to use "Resource
>>Manager" that can control the amount of Workflows running concurrently on
>>a node. Basically, my set up is like this:
>>
>>
>>- I have a Workflow Manager Server configured to run at most 10 Workflow
>>Events concurrently.
>>- I have a bunch of files sitting on a RabbitMQ queue.
>>- I have a listener that is continually monitoring this particular queue.
>>As soon as it sees a file in this queue, it will trigger a Workflow
>>event.
>>
>>
>>Under this scenario, normally if I have 30 files in the RabbitMQ, my
>>listener will trigger 30 Workflow events. However, the Workflow Server
>>would put 20 of these events in its repository queue while the other 10
>>are running.
>>
>>
>>What I want some "Resource Manager" component to do is just run the 10
>>Workflow events only and have the other 20 sitting in the RabbitMQ. As
>>soon as 1 event has finished, then trigger another workflow event to keep
>>the number of concurrent workflows running
>> at 10.
>>Eventually, we will want this "Resource Manager" component to be able to
>>point to another node, node 2, where it can decide to trigger another set
>>of 10 Workflow events while node 1 is still running it's 10 Workflow
>>events.
>>
>>
>>Hope this makes sense. Any helpful tips would be much appreciated.
>>
>>
>>Thanks,
>>Mike
>