oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Verma, Rishi (388J)" <Rishi.Ve...@jpl.nasa.gov>
Subject Re: Sharing workflow task metadata when using a resource manager
Date Tue, 25 Oct 2011 00:45:24 GMT
Hey Sheryl,

Thanks for the response!

Yeah, the workflow-manager log actually already spits out a lot of workflow task instance
info whenever it submits a task to a resource manager node.
It shows (for each task invoked):
* 'task.intance.class' name
* 'task.config' parameters
* 'task.metadata' contents

I did some tests, and concluded (from viewing task.metadata) that all task metadata fed into
the first task of my workflow (ie. when passing in met via: wmgr-client ... --key name value...)
showed up in the 'task.metadata' for all tasks in my workflow. However, if my first task added
new generated metadata, it would not show up in the 'task.metadata' of subsequent tasks being
called. The latter issue only occurs when using a resource manager for task execution.

Thanks for the tip though, I will keep this in mind while trying out some new strategies.


On Oct 24, 2011, at 5:18 PM, Sheryl John wrote:

> Hi Rishi,
> Would getting workflowInstance metadata help you debug this problem ( using
> command-line option for the wmgr-client)? This will not give you the job
> metadata of course, but it gives out the job ids.
> And you could probably use the job id for checking out the metadata for
> that respective job. I'm not sure if there is code for this.
> On Mon, Oct 24, 2011 at 4:52 PM, Verma, Rishi (388J) <
> Rishi.Verma@jpl.nasa.gov> wrote:
>> Hi all,
>> I'm trying to pass generated metadata from a workflow task to another
>> workflow task (both in the same workflow) when using a resource manager to
>> execute workflow task jobs.
>> I've done this before successfully when workflow tasks within a given
>> workflow are run locally (by the workflow manager itself) but when I point
>> workflow manager to have tasks execute through a resource manager, my
>> generated metadata does not seem to transfer from one task to the next.
>> By "generated" metadata, I mean metadata that is added within the "run"
>> method of an implemented WorkflowTaskInstance. It's worth noting though,
>> that metadata passed into the initial XmlRpc call of a workflow task seems
>> to be passed to all subsequent tasks just fine. Just not generated metadata
>> - which passes only when not using a resource manager.
>> I'm trying to ascertain if this issue is a bug or not. To find out, could
>> someone elaborate a bit on which resmgr (or other) classes would include
>> code that actually shows metadata for a job being passed through a remotely
>> running job? I've been trying to find such code within the codebase but have
>> not come across it yet.
>> Thanks!
>> rishi
> -- 
> -Sheryl

View raw message