oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Sharing workflow task metadata when using a resource manager
Date Thu, 27 Oct 2011 01:44:48 GMT
Hi Rishi,

Thanks. Yep that's the problem below (and why Jenkins has been screaming at me for weeks).
I broke the build 
a few weeks ago when I committed portions of OODT-310. I want to wrap up OODT-310 and need
to get to it, 
but in the meanwhile, if you can use a revision prior to those commits (or better yet, a revision
right around 
when I wrapped up OODT-70), or even 0.3, that will get you on the right track for now.

Thanks!

Cheers,
Chris


On Oct 26, 2011, at 3:17 PM, Verma, Rishi (388J) wrote:

> Hi Chris,
> 
> Thanks for the code snippet.
> 
> I'm running my XmlRpcBatchStub on the same host as my workflow manager, so I don't think
that is the issue.
> 
> I looked at my XmlRpcBatchStub log however, and found the following exception being issued
whenever one of my tasks was sent to this batch stub:
> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.ClassCastException:
org.apache.oodt.cas.workflow.engine.SequentialProcessor cannot be cast to org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngine$ThreadedProcessor
> 	at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
> 	at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
> 	at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
> 	at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
> 	at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
> 	at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
> 	at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.updateMetadataForWorkflow(XmlRpcWorkflowManagerClient.java:424)
> 	at org.apache.oodt.cas.workflow.structs.TaskJob.updateMetadata(TaskJob.java:137)
> 	at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:113)
> 	at org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
> 	at java.lang.Thread.run(Thread.java:619)
> 
> I'm using workflow-manager 0.4-SNAPSHOT (not the latest trunk). Off the bat, does anything
look strange to you? It seems from the exception, that the workflow met is definitely not
being properly passed to the batch stub.
> 
> I know a lot of updates have been made to WM lately, so I tried to build the latest trunk
version of 0.4-SNAPSHOT; however, it currently fails the 'TestXmlRpcWorkflowManagerClient'
test with:
> java.text.ParseException: An exception occurred because the input date/time string was
not at least 24 characters in length.
>        at org.apache.oodt.commons.util.DateConvert.isoParse(DateConvert.java:165)
>        at org.apache.oodt.cas.workflow.structs.WorkflowInstance.setCurrentTaskStartDateTimeIsoStr(WorkflowInstance.java:339)
> 
> I'm going to look more into it and investigate, but maybe I will try deploying with 0.3
OODT and see if the problem persists.
> 
> Thanks!
> rishi
> 
> 
> 
> 
> 
> 
> On Oct 24, 2011, at 9:07 PM, Mattmann, Chris A (388J) wrote:
> 
>> Hi Rishi,
>> 
>> The metadata context should be persisted, based on updates made during the tasks
themselves. If you have a look at 
>> TaskJob [1], you'll see the last step in the execute(JobInput in) method is:
>> 
>> {code}
>>    // now we have to update the workflow manager with the metadata
>>       // that may have been updated
>>       updateMetadata(taskInput.getDynMetadata());
>> {code}
>> 
>> which should call back to the parent workflow manager and update the workflow instance
>> with the current metadata. If you're not seeing this it's possible that your XmlRpcBatchStub

>> on your executing node cannot see back to the parent WM running on host:port.
>> 
>> Is this the case?
>> 
>> Cheers,
>> Chris
>> 
>> On Oct 24, 2011, at 4:52 PM, Verma, Rishi (388J) wrote:
>> 
>>> Hi all,
>>> 
>>> I'm trying to pass generated metadata from a workflow task to another workflow
task (both in the same workflow) when using a resource manager to execute workflow task jobs.

>>> 
>>> I've done this before successfully when workflow tasks within a given workflow
are run locally (by the workflow manager itself) but when I point workflow manager to have
tasks execute through a resource manager, my generated metadata does not seem to transfer
from one task to the next.
>>> 
>>> By "generated" metadata, I mean metadata that is added within the "run" method
of an implemented WorkflowTaskInstance. It's worth noting though, that metadata passed into
the initial XmlRpc call of a workflow task seems to be passed to all subsequent tasks just
fine. Just not generated metadata - which passes only when not using a resource manager.
>>> 
>>> I'm trying to ascertain if this issue is a bug or not. To find out, could someone
elaborate a bit on which resmgr (or other) classes would include code that actually shows
metadata for a job being passed through a remotely running job? I've been trying to find such
code within the codebase but have not come across it yet.
>>> 
>>> Thanks!
>>> rishi
>>> 
>> 
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 
> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message