oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renu Kanakamedala <kanak...@usc.edu>
Subject Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.
Date Thu, 06 Nov 2014 05:51:12 GMT
*wmgr script *

http://pastebin.com/zE7PWnXL

*resmgr batch_stub*

http://pastebin.com/gPUvbE9e

- Renu


On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann <mattmann@usc.edu>
wrote:

> Great. One last question.
>
> Can you show me the contents (paste.bin) of your wmgr
> script and your batch_stub script?
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Adjunct Associate Professor, Computer Science Department
> University of Southern California
> Los Angeles, CA 90089 USA
> Email: mattmann@usc.edu
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Renu Kanakamedala <kanakame@usc.edu>
> Date: Wednesday, November 5, 2014 at 10:20 PM
> To: Chris Mattmann <mattmann@usc.edu>
> Cc: Meghamsh Utkur <utkur@usc.edu>, "dev@oodt.apache.org"
> <dev@oodt.apache.org>, Preethi Ramesh <pramesh@usc.edu>, Gouthami
> Kondakindi <kondakin@usc.edu>
> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> parse null Error.
>
> >renu@ubuntu:~$ echo $WORKFLOW_HOME
> >/home/renu/oodt/workflow/
> >renu@ubuntu:~$ echo $FILEMGR_HOME
> >/home/renu/oodt/filemgr
> >renu@ubuntu:~$ echo $WORKFLOW_URL
> >http://localhost:9001
> >renu@ubuntu:~$ echo $FILEMGR_URL
> >http://localhost:9000
> >renu@ubuntu:~$ echo $PGE_ROOT
> >/home/renu/oodt/pge/
> >
> >renu@ubuntu:~$ echo $CRAWLER_HOME
> >/home/renu/oodt/crawler/
> >
> >
> >
> >
> >
> >
> >
> >On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
> ><mattmann@usc.edu> wrote:
> >
> >Right, which is why I asked to specifically echo the value
> >of all environment variables so we would be able to see the
> >values. Can Renu please show the echo values?
> >
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >Chris Mattmann, Ph.D.
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California
> >Los Angeles, CA 90089 USA
> >Email: mattmann@usc.edu
> >WWW: http://sunset.usc.edu/~mattmann/
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >-----Original Message-----
> >From: Meghamsh Utkur <utkur@usc.edu>
> >Date: Wednesday, November 5, 2014 at 9:35 PM
> >To: Renu Kanakamedala <kanakame@usc.edu>
> >Cc: Chris Mattmann <mattmann@usc.edu>, "dev@oodt.apache.org"
> ><dev@oodt.apache.org>, Preethi Ramesh <pramesh@usc.edu>, Gouthami
> >Kondakindi <kondakin@usc.edu>
> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >parse null Error.
> >
> >>Professor,
> >>
> >>
> >>To add to what Renu has said, I went through the source code and I
> >>suspect that the value here in this method is passed as null.
> >>
> >>XmlFilePgeConfigBuilder.java
> >>
> >>1. private String fillIn(String value,Metadata inputMetadata,boolean
> >>envReplaceRecur)
> >>2. throwsException {
> >>3. try{
> >>4. while((value =PathUtils.doDynamicReplacement(value,
> >>inputMetadata)).contains("[")&& envReplaceRecur);
> >>5. if(value.toUpperCase()
> >>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
> >>7. value
> >>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
> >>URL(inputMetadata
> >>8.
> >>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.p
> >>a
> >>rseSqlQueryMethod(value)));
> >>9. return value;
> >>10. }catch(Exception e){
> >>11. thrownew Exception("Failed to parse value: "+ value, e);
> >>12. }
> >>13. }
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
> >><kanakame@usc.edu> wrote:
> >>
> >>Hello Professor,
> >>
> >>
> >>All the environment variables present in tasks.xml i.e.
> >>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
> >>
> >>
> >>Please let us know how to proceed.
> >>
> >>
> >>Regards,
> >>Renu
> >>
> >>
> >>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
> >><mattmann@usc.edu> wrote:
> >>
> >>Can you please confirm that all of the bracketed environment
> >>variables below do in fact exist in the shell that you are
> >>running ./oodt start in?
> >>
> >>E.g., please echo $ENV_VAR below and show their values.
> >>
> >>Cheers,
> >>Chris
> >>
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>Chris Mattmann, Ph.D.
> >>Adjunct Associate Professor, Computer Science Department
> >>University of Southern California
> >>Los Angeles, CA 90089 USA
> >>Email: mattmann@usc.edu
> >>WWW: http://sunset.usc.edu/~mattmann/
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
> >>
> >>
> >>
> >>-----Original Message-----
> >>From: Renu Kanakamedala <kanakame@usc.edu>
> >>Date: Wednesday, November 5, 2014 at 8:18 PM
> >>To: Chris Mattmann <mattmann@usc.edu>
> >>Cc: "dev@oodt.apache.org" <dev@oodt.apache.org>, Preethi Ramesh
> >><pramesh@usc.edu>, Gouthami Kondakindi <kondakin@usc.edu>
> >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >>parse null Error.
> >>
> >>>Please find below tasks.xml file.
> >>>
> >>>
> >>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
> >>><!--
> >>>  TODO: Add some examples
> >>>-->
> >>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
> >>>
> >>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGE
> >>>T
> >>>a
> >>>sk">
> >>>
> >>>  <conditions/>
> >>>
> >>>  <configuration>
> >>>     <property name="PGETask_Name" value="FileConcatenator"/>
> >>>     <property name="PGETask_ConfigFilePath"
> >>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
> >>>envReplace="true"/>
> >>>     <property name="PGETask_DumpMetadata" value="true"/>
> >>>     <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
> >>>envReplace="true" />
> >>>     <property name="PCS_FileManagerUrl"     value="[FILEMGR_URL]"
> >>>envReplace="true"/>
> >>>     <property name="PCS_MetFileExtension" value="met"/>
> >>>     <property name="PCS_ClientTransferServiceFactory"
> >>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
> >>>"
> >>>/
> >>>>
> >>>     <property name="PCS_ActionRepoFile"
> >>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
> >>>envReplace="true"/>
> >>>  </configuration>
> >>>
> >>>  <requiredMetFields>
> >>>     <metfield name="RunID"/>
> >>>  </requiredMetFields>
> >>>
> >>></task>
> >>></cas:tasks>
> >>>
> >>>
> >>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
> >>><mattmann@usc.edu> wrote:
> >>>
> >>>Hi Renu,
> >>>
> >>>Can you please show a paste.bin of your tasks.xml?
> >>>
> >>>Cheers,
> >>>Chris
> >>>
> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>Chris Mattmann, Ph.D.
> >>>Adjunct Associate Professor, Computer Science Department
> >>>University of Southern California
> >>>Los Angeles, CA 90089 USA
> >>>Email: mattmann@usc.edu
> >>>WWW: http://sunset.usc.edu/~mattmann/
> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>
> >>>
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: Renu Kanakamedala <kanakame@usc.edu>
> >>>Date: Wednesday, November 5, 2014 at 7:55 PM
> >>>To: Chris Mattmann <mattmann@usc.edu>
> >>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >>>parse
> >>>null Error.
> >>>
> >>>>Hello Professor,
> >>
> >>>>
> >>>>
> >>>>When i am trying to run file concatenator CAS PGE example using the
> >>>>following command.
> >>>>
> >>>>
> >>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
> >>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
> >>>>
> >>>>
> >>>>
> >>>>The message shown after executing this command is "Sending Event
> >>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
> >>>>stub.
> >>>>
> >>>>
> >>>>Batch Stub:
> >>>>Caused by:
> >>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc
> >>>>e
> >>>>p
> >>>>t
> >>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> >>>>value: null
> >>>>at
> >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> >>>>at
> >>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> >>>>... 2 more
> >>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> >>>>parse value: null
> >>>>at
> >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
> >>>>C
> >>>>o
> >>>>n
> >>>>figBuilder.java:87)
> >>>>at
> >>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance
> >>>>.
> >>>>j
> >>>>a
> >>>>va:230)
> >>>>at
> >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> >>>>... 3 more
> >>>>Caused by: java.lang.Exception: Failed to parse value: null
> >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >>>>at
> >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge
> >>>>C
> >>>>o
> >>>>n
> >>>>figBuilder.java:77)
> >>>>... 5 more
> >>>>Caused by: java.lang.NullPointerException
> >>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >>>>at java.util.regex.Matcher.reset(Matcher.java:308)
> >>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
> >>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >>>>at
> >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplac
> >>>>e
> >>>>m
> >>>>e
> >>>>nt(PathUtils.java:321)
> >>>>at
> >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUt
> >>>>i
> >>>>l
> >>>>s
> >>>>.java:96)
> >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
> >>>>... 7 more
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>RES mngr:
> >>>>
> >>>>
> >>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
> >>>>addJob
> >>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
> >>>>Nov 05, 2014 5:26:30 PM
> >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
> >>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
> >>>>Scheduling for execution
> >>>>Nov 05, 2014 5:26:30 PM
> >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
> >>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: ProxySelector Request for
> >>>
> >>>
> >>
> >>
> >>>>http://localhost:2001/ <http://localhost:2001/>
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: Proxy used: DIRECT
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>writeRequests
> >>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
> >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>getInputStream
> >>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
> >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>>text/xml}{Content-Length: 121}
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: ProxySelector Request for
> >>>>http://localhost:2001/ <http://localhost:2001/>
> >>
> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>plainConnect
> >>>>FINEST: Proxy used: DIRECT
> >>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>writeRequests
> >>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
> >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
> >>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
> >>>>getInputStream
> >>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
> >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>>text/xml}{Content-Length: 121}
> >>>>Nov 05, 2014 5:26:32 PM
> >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
> >>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00'
> >>>>:
> >>>>batchstub.executeJob returned false
> >>>>java.lang.Exception: batchstub.executeJob returned false
> >>>>at
> >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBat
> >>>>c
> >>>>h
> >>>>M
> >>>>grProxy.java:125)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>Workflow:
> >>>>
> >>>>
> >>>>Nov 05, 2014 5:26:27 PM
> >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>>checkTaskRequiredMetadata
> >>>>INFO: All required metadata fields present for task: [FileConcatenator]
> >>>>Nov 05, 2014 5:26:28 PM
> >>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
> >>>>submitJob
> >>>>FINEST: [{job.queueName=high,
> >>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>>>job.name <http://job.name> <http://job.name> <http://job.name>
> >>>><http://job.name>=urn:oodt:FileConcatenator,
> >
> >
> >>>>job.id <http://job.id> <http://job.id> <http://job.id>
> >>>><http://job.id>=, job.status=,
> >>>>job.load=2,
> >>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.File
> >>>>C
> >>>>o
> >>>>n
> >>>>catenatorPGETask,
> >>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/
> >>>>P
> >>>>G
> >>>>E
> >>>>Config.xml,
> >>>>
> >>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransf
> >>>>e
> >>>>r
> >>>>.
> >>>>LocalDataTransferFactory,
> >>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.x
> >>>>m
> >>>>l
> >>>>,
> >>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >>>>PCS_WorkflowManagerUrl=http://localhost:9001,
> >>>> PCS_FileManagerUrl=http://localhost:9000,
> >>>>PGETask_Name=FileConcatenator},
> >>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >>>>WorkflowManagerUrl=[http://ubuntu:9001],
> >>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> >>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
> >>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
> >>>>Nov 05, 2014 5:26:39 PM
> >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>>stop
> >>>>WARNING: Attempt to kill current resmgr job:
> >>>>[2014-11-05T17:26:28.785-08:00]: failed
> >>>>
> >>>>
> >>>>When i searched online i found one of your mail archives where you had
> >>>>mentioned to change tasks.xml
> >>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
> >>>>
> >>>>
> >>>>
> >>>>It did not work. Please let us know how to proceed.
> >>>>
> >>>>
> >>>>
> >>>>Regards,
> >>>>Renu
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>--
> >
> >
> >>Meghamsh UtkurGraduate Student,University of Southern California
> >>utkur@usc.edu
> >>
> >>(503)881-4791 <tel:%28503%29881-4791>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message