oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Foster <holeno...@me.com>
Subject Re: Sharing workflow task metadata when using a resource manager
Date Tue, 25 Oct 2011 01:17:40 GMT
if you are using CAS-PGE then in your pgeconfig.xml file you can specify on a any custom metadata
element the attribute workflowMet='true' and it will pass through to all the following tasks
in the workflow... for example: <metadata key='filename' val='data.dat' workflowMet='true'/>

-Brian

On Oct 24, 2011, at 5:45 PM, "Verma, Rishi (388J)" <Rishi.Verma@jpl.nasa.gov> wrote:

> Hey Sheryl,
> 
> Thanks for the response!
> 
> Yeah, the workflow-manager log actually already spits out a lot of workflow task instance
info whenever it submits a task to a resource manager node.
> It shows (for each task invoked):
> * 'task.intance.class' name
> * 'task.config' parameters
> * 'task.metadata' contents
> 
> I did some tests, and concluded (from viewing task.metadata) that all task metadata fed
into the first task of my workflow (ie. when passing in met via: wmgr-client ... --key name
value...) showed up in the 'task.metadata' for all tasks in my workflow. However, if my first
task added new generated metadata, it would not show up in the 'task.metadata' of subsequent
tasks being called. The latter issue only occurs when using a resource manager for task execution.
> 
> Thanks for the tip though, I will keep this in mind while trying out some new strategies.
> 
> rishi
> 
> On Oct 24, 2011, at 5:18 PM, Sheryl John wrote:
> 
>> Hi Rishi,
>> 
>> Would getting workflowInstance metadata help you debug this problem ( using
>> command-line option for the wmgr-client)? This will not give you the job
>> metadata of course, but it gives out the job ids.
>> And you could probably use the job id for checking out the metadata for
>> that respective job. I'm not sure if there is code for this.
>> 
>> On Mon, Oct 24, 2011 at 4:52 PM, Verma, Rishi (388J) <
>> Rishi.Verma@jpl.nasa.gov> wrote:
>> 
>>> Hi all,
>>> 
>>> I'm trying to pass generated metadata from a workflow task to another
>>> workflow task (both in the same workflow) when using a resource manager to
>>> execute workflow task jobs.
>>> 
>>> I've done this before successfully when workflow tasks within a given
>>> workflow are run locally (by the workflow manager itself) but when I point
>>> workflow manager to have tasks execute through a resource manager, my
>>> generated metadata does not seem to transfer from one task to the next.
>>> 
>>> By "generated" metadata, I mean metadata that is added within the "run"
>>> method of an implemented WorkflowTaskInstance. It's worth noting though,
>>> that metadata passed into the initial XmlRpc call of a workflow task seems
>>> to be passed to all subsequent tasks just fine. Just not generated metadata
>>> - which passes only when not using a resource manager.
>>> 
>>> I'm trying to ascertain if this issue is a bug or not. To find out, could
>>> someone elaborate a bit on which resmgr (or other) classes would include
>>> code that actually shows metadata for a job being passed through a remotely
>>> running job? I've been trying to find such code within the codebase but have
>>> not come across it yet.
>>> 
>>> Thanks!
>>> rishi
>>> 
>>> 
>> 
>> 
>> -- 
>> -Sheryl
> 

Mime
View raw message