oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Verma, Rishi (388J)" <Rishi.Ve...@jpl.nasa.gov>
Subject Re: Sharing workflow task metadata when using a resource manager
Date Fri, 28 Oct 2011 01:14:34 GMT
Hey Chris,

I tried 0.3, and it worked just fine (generated workflow met passed down to other tasks when
using resmgr batchstubs). Thanks for the suggestion!

rishi

On Oct 26, 2011, at 6:44 PM, Mattmann, Chris A (388J) wrote:

> Hi Rishi,
> 
> Thanks. Yep that's the problem below (and why Jenkins has been screaming at me for weeks).
I broke the build 
> a few weeks ago when I committed portions of OODT-310. I want to wrap up OODT-310 and
need to get to it, 
> but in the meanwhile, if you can use a revision prior to those commits (or better yet,
a revision right around 
> when I wrapped up OODT-70), or even 0.3, that will get you on the right track for now.
> 
> Thanks!
> 
> Cheers,
> Chris
> 
> 
> On Oct 26, 2011, at 3:17 PM, Verma, Rishi (388J) wrote:
> 
>> Hi Chris,
>> 
>> Thanks for the code snippet.
>> 
>> I'm running my XmlRpcBatchStub on the same host as my workflow manager, so I don't
think that is the issue.
>> 
>> I looked at my XmlRpcBatchStub log however, and found the following exception being
issued whenever one of my tasks was sent to this batch stub:
>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: java.lang.ClassCastException:
org.apache.oodt.cas.workflow.engine.SequentialProcessor cannot be cast to org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngine$ThreadedProcessor
>> 	at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
>> 	at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
>> 	at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
>> 	at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
>> 	at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
>> 	at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
>> 	at org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.updateMetadataForWorkflow(XmlRpcWorkflowManagerClient.java:424)
>> 	at org.apache.oodt.cas.workflow.structs.TaskJob.updateMetadata(TaskJob.java:137)
>> 	at org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:113)
>> 	at org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229)
>> 	at java.lang.Thread.run(Thread.java:619)
>> 
>> I'm using workflow-manager 0.4-SNAPSHOT (not the latest trunk). Off the bat, does
anything look strange to you? It seems from the exception, that the workflow met is definitely
not being properly passed to the batch stub.
>> 
>> I know a lot of updates have been made to WM lately, so I tried to build the latest
trunk version of 0.4-SNAPSHOT; however, it currently fails the 'TestXmlRpcWorkflowManagerClient'
test with:
>> java.text.ParseException: An exception occurred because the input date/time string
was not at least 24 characters in length.
>>       at org.apache.oodt.commons.util.DateConvert.isoParse(DateConvert.java:165)
>>       at org.apache.oodt.cas.workflow.structs.WorkflowInstance.setCurrentTaskStartDateTimeIsoStr(WorkflowInstance.java:339)
>> 
>> I'm going to look more into it and investigate, but maybe I will try deploying with
0.3 OODT and see if the problem persists.
>> 
>> Thanks!
>> rishi
>> 
>> 
>> 
>> 
>> 
>> 
>> On Oct 24, 2011, at 9:07 PM, Mattmann, Chris A (388J) wrote:
>> 
>>> Hi Rishi,
>>> 
>>> The metadata context should be persisted, based on updates made during the tasks
themselves. If you have a look at 
>>> TaskJob [1], you'll see the last step in the execute(JobInput in) method is:
>>> 
>>> {code}
>>>   // now we have to update the workflow manager with the metadata
>>>      // that may have been updated
>>>      updateMetadata(taskInput.getDynMetadata());
>>> {code}
>>> 
>>> which should call back to the parent workflow manager and update the workflow
instance
>>> with the current metadata. If you're not seeing this it's possible that your
XmlRpcBatchStub 
>>> on your executing node cannot see back to the parent WM running on host:port.
>>> 
>>> Is this the case?
>>> 
>>> Cheers,
>>> Chris
>>> 
>>> On Oct 24, 2011, at 4:52 PM, Verma, Rishi (388J) wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> I'm trying to pass generated metadata from a workflow task to another workflow
task (both in the same workflow) when using a resource manager to execute workflow task jobs.

>>>> 
>>>> I've done this before successfully when workflow tasks within a given workflow
are run locally (by the workflow manager itself) but when I point workflow manager to have
tasks execute through a resource manager, my generated metadata does not seem to transfer
from one task to the next.
>>>> 
>>>> By "generated" metadata, I mean metadata that is added within the "run" method
of an implemented WorkflowTaskInstance. It's worth noting though, that metadata passed into
the initial XmlRpc call of a workflow task seems to be passed to all subsequent tasks just
fine. Just not generated metadata - which passes only when not using a resource manager.
>>>> 
>>>> I'm trying to ascertain if this issue is a bug or not. To find out, could
someone elaborate a bit on which resmgr (or other) classes would include code that actually
shows metadata for a job being passed through a remotely running job? I've been trying to
find such code within the codebase but have not come across it yet.
>>>> 
>>>> Thanks!
>>>> rishi
>>>> 
>>> 
>>> 
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW:   http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> 
>> 
> 
> 
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 


Mime
View raw message