oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renu Kanakamedala <kanak...@usc.edu>
Subject Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.
Date Thu, 06 Nov 2014 05:20:50 GMT
renu@ubuntu:~$ echo $WORKFLOW_HOME
/home/renu/oodt/workflow/
renu@ubuntu:~$ echo $FILEMGR_HOME
/home/renu/oodt/filemgr
renu@ubuntu:~$ echo $WORKFLOW_URL
http://localhost:9001
renu@ubuntu:~$ echo $FILEMGR_URL
http://localhost:9000
renu@ubuntu:~$ echo $PGE_ROOT
/home/renu/oodt/pge/
renu@ubuntu:~$ echo $CRAWLER_HOME
/home/renu/oodt/crawler/



On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann <mattmann@usc.edu>
wrote:

> Right, which is why I asked to specifically echo the value
> of all environment variables so we would be able to see the
> values. Can Renu please show the echo values?
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Adjunct Associate Professor, Computer Science Department
> University of Southern California
> Los Angeles, CA 90089 USA
> Email: mattmann@usc.edu
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
> -----Original Message-----
> From: Meghamsh Utkur <utkur@usc.edu>
> Date: Wednesday, November 5, 2014 at 9:35 PM
> To: Renu Kanakamedala <kanakame@usc.edu>
> Cc: Chris Mattmann <mattmann@usc.edu>, "dev@oodt.apache.org"
> <dev@oodt.apache.org>, Preethi Ramesh <pramesh@usc.edu>, Gouthami
> Kondakindi <kondakin@usc.edu>
> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> parse null Error.
>
> >Professor,
> >
> >
> >To add to what Renu has said, I went through the source code and I
> >suspect that the value here in this method is passed as null.
> >
> >XmlFilePgeConfigBuilder.java
> >
> >1. private String fillIn(String value,Metadata inputMetadata,boolean
> >envReplaceRecur)
> >2. throwsException {
> >3. try{
> >4. while((value =PathUtils.doDynamicReplacement(value,
> >inputMetadata)).contains("[")&& envReplaceRecur);
> >5. if(value.toUpperCase()
> >6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
> >7. value
> >=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
> >URL(inputMetadata
> >8.
> >.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.pa
> >rseSqlQueryMethod(value)));
> >9. return value;
> >10. }catch(Exception e){
> >11. thrownew Exception("Failed to parse value: "+ value, e);
> >12. }
> >13. }
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
> ><kanakame@usc.edu> wrote:
> >
> >Hello Professor,
> >
> >
> >All the environment variables present in tasks.xml i.e.
> >$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
> >
> >
> >Please let us know how to proceed.
> >
> >
> >Regards,
> >Renu
> >
> >
> >On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
> ><mattmann@usc.edu> wrote:
> >
> >Can you please confirm that all of the bracketed environment
> >variables below do in fact exist in the shell that you are
> >running ./oodt start in?
> >
> >E.g., please echo $ENV_VAR below and show their values.
> >
> >Cheers,
> >Chris
> >
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >Chris Mattmann, Ph.D.
> >Adjunct Associate Professor, Computer Science Department
> >University of Southern California
> >Los Angeles, CA 90089 USA
> >Email: mattmann@usc.edu
> >WWW: http://sunset.usc.edu/~mattmann/
> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >-----Original Message-----
> >From: Renu Kanakamedala <kanakame@usc.edu>
> >Date: Wednesday, November 5, 2014 at 8:18 PM
> >To: Chris Mattmann <mattmann@usc.edu>
> >Cc: "dev@oodt.apache.org" <dev@oodt.apache.org>, Preethi Ramesh
> ><pramesh@usc.edu>, Gouthami Kondakindi <kondakin@usc.edu>
> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
> >parse null Error.
> >
> >>Please find below tasks.xml file.
> >>
> >>
> >><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
> >><!--
> >>  TODO: Add some examples
> >>-->
> >><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
> >>
> >>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGET
> >>a
> >>sk">
> >>
> >>  <conditions/>
> >>
> >>  <configuration>
> >>     <property name="PGETask_Name" value="FileConcatenator"/>
> >>     <property name="PGETask_ConfigFilePath"
> >>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
> >>envReplace="true"/>
> >>     <property name="PGETask_DumpMetadata" value="true"/>
> >>     <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
> >>envReplace="true" />
> >>     <property name="PCS_FileManagerUrl"     value="[FILEMGR_URL]"
> >>envReplace="true"/>
> >>     <property name="PCS_MetFileExtension" value="met"/>
> >>     <property name="PCS_ClientTransferServiceFactory"
> >>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"
> >>/
> >>>
> >>     <property name="PCS_ActionRepoFile"
> >>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
> >>  </configuration>
> >>
> >>  <requiredMetFields>
> >>     <metfield name="RunID"/>
> >>  </requiredMetFields>
> >>
> >></task>
> >></cas:tasks>
> >>
> >>
> >>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
> >><mattmann@usc.edu> wrote:
> >>
> >>Hi Renu,
> >>
> >>Can you please show a paste.bin of your tasks.xml?
> >>
> >>Cheers,
> >>Chris
> >>
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>Chris Mattmann, Ph.D.
> >>Adjunct Associate Professor, Computer Science Department
> >>University of Southern California
> >>Los Angeles, CA 90089 USA
> >>Email: mattmann@usc.edu
> >>WWW: http://sunset.usc.edu/~mattmann/
> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
> >>
> >>
> >>
> >>-----Original Message-----
> >>From: Renu Kanakamedala <kanakame@usc.edu>
> >>Date: Wednesday, November 5, 2014 at 7:55 PM
> >>To: Chris Mattmann <mattmann@usc.edu>
> >>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
> >>null Error.
> >>
> >>>Hello Professor,
> >
> >>>
> >>>
> >>>When i am trying to run file concatenator CAS PGE example using the
> >>>following command.
> >>>
> >>>
> >>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
> >>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
> >>>
> >>>
> >>>
> >>>The message shown after executing this command is "Sending Event
> >>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
> >>>stub.
> >>>
> >>>
> >>>Batch Stub:
> >>>Caused by:
> >>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExce
> >>>p
> >>>t
> >>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
> >>>value: null
> >>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
> >>>at
> >>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
> >>>... 2 more
> >>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
> >>>parse value: null
> >>>at
> >>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
> >>>o
> >>>n
> >>>figBuilder.java:87)
> >>>at
> >>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.
> >>>j
> >>>a
> >>>va:230)
> >>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
> >>>... 3 more
> >>>Caused by: java.lang.Exception: Failed to parse value: null
> >>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
> >>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
> >>>at
> >>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
> >>>o
> >>>n
> >>>figBuilder.java:77)
> >>>... 5 more
> >>>Caused by: java.lang.NullPointerException
> >>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
> >>>at java.util.regex.Matcher.reset(Matcher.java:308)
> >>>at java.util.regex.Matcher.<init>(Matcher.java:228)
> >>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
> >>>at
> >>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplace
> >>>m
> >>>e
> >>>nt(PathUtils.java:321)
> >>>at
> >>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUti
> >>>l
> >>>s
> >>>.java:96)
> >>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
> >>>... 7 more
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>RES mngr:
> >>>
> >>>
> >>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
> >>>addJob
> >>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
> >>>Nov 05, 2014 5:26:30 PM
> >>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
> >>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
> >>>Scheduling for execution
> >>>Nov 05, 2014 5:26:30 PM
> >>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
> >>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: ProxySelector Request for
> >>
> >>
> >
> >
> >>>http://localhost:2001/ <http://localhost:2001/>
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: Proxy used: DIRECT
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>writeRequests
> >>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
> >>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>> q=.2}{Connection: keep-alive}{Content-Length: 105}
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>getInputStream
> >>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
> >>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>text/xml}{Content-Length: 121}
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: ProxySelector Request for
> >>>http://localhost:2001/ <http://localhost:2001/>
> >
> >>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
> >>>plainConnect
> >>>FINEST: Proxy used: DIRECT
> >>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
> >>>writeRequests
> >>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
> >>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
> >>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
> >>>text/html, image/gif, image/jpeg, *; q=.2, */*;
> >>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
> >>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
> >>>getInputStream
> >>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
> >>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
> >>>text/xml}{Content-Length: 121}
> >>>Nov 05, 2014 5:26:32 PM
> >>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
> >>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
> >>>batchstub.executeJob returned false
> >>>java.lang.Exception: batchstub.executeJob returned false
> >>>at
> >>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatc
> >>>h
> >>>M
> >>>grProxy.java:125)
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>Workflow:
> >>>
> >>>
> >>>Nov 05, 2014 5:26:27 PM
> >>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>checkTaskRequiredMetadata
> >>>INFO: All required metadata fields present for task: [FileConcatenator]
> >>>Nov 05, 2014 5:26:28 PM
> >>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
> >>>submitJob
> >>>FINEST: [{job.queueName=high,
> >>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
> >>
> >>
> >
> >
> >
> >
> >>>job.name <http://job.name> <http://job.name>
> >>><http://job.name>=urn:oodt:FileConcatenator,
> >>>job.id <http://job.id> <http://job.id> <http://job.id>=,
job.status=,
> >>>job.load=2,
> >>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
> >>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileC
> >>>o
> >>>n
> >>>catenatorPGETask,
> >>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/P
> >>>G
> >>>E
> >>>Config.xml,
> >>>
> >>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfe
> >>>r
> >>>.
> >>>LocalDataTransferFactory,
> >>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xm
> >>>l
> >>>,
> >>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
> >>>PCS_WorkflowManagerUrl=http://localhost:9001,
> >>> PCS_FileManagerUrl=http://localhost:9000,
> >>>PGETask_Name=FileConcatenator},
> >>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
> >>>WorkflowManagerUrl=[http://ubuntu:9001],
> >>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
> >>> ProcessingNode=[ubuntu], RunID=[testNumber1],
> >>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
> >>>Nov 05, 2014 5:26:39 PM
> >>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
> >>>stop
> >>>WARNING: Attempt to kill current resmgr job:
> >>>[2014-11-05T17:26:28.785-08:00]: failed
> >>>
> >>>
> >>>When i searched online i found one of your mail archives where you had
> >>>mentioned to change tasks.xml
> >>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
> >>>
> >>>
> >>>
> >>>It did not work. Please let us know how to proceed.
> >>>
> >>>
> >>>
> >>>Regards,
> >>>Renu
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >--
> >Meghamsh UtkurGraduate Student,University of Southern California
> >utkur@usc.edu
> >
> >(503)881-4791
> >
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message