oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Alan Mattmann <mattm...@usc.edu>
Subject Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.
Date Thu, 06 Nov 2014 04:38:19 GMT
Right, which is why I asked to specifically echo the value
of all environment variables so we would be able to see the
values. Can Renu please show the echo values?

++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Meghamsh Utkur <utkur@usc.edu>
Date: Wednesday, November 5, 2014 at 9:35 PM
To: Renu Kanakamedala <kanakame@usc.edu>
Cc: Chris Mattmann <mattmann@usc.edu>, "dev@oodt.apache.org"
<dev@oodt.apache.org>, Preethi Ramesh <pramesh@usc.edu>, Gouthami
Kondakindi <kondakin@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.

>Professor,
>
>
>To add to what Renu has said, I went through the source code and I
>suspect that the value here in this method is passed as null.
>
>XmlFilePgeConfigBuilder.java
>
>1. private String fillIn(String value,Metadata inputMetadata,boolean
>envReplaceRecur)
>2. throwsException {
>3. try{
>4. while((value =PathUtils.doDynamicReplacement(value,
>inputMetadata)).contains("[")&& envReplaceRecur);
>5. if(value.toUpperCase()
>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
>7. value 
>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
>URL(inputMetadata
>8. 
>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.pa
>rseSqlQueryMethod(value)));
>9. return value;
>10. }catch(Exception e){
>11. thrownew Exception("Failed to parse value: "+ value, e);
>12. }
>13. }
>
>
>
>
>
>
>
>
>
>
>
>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
><kanakame@usc.edu> wrote:
>
>Hello Professor,
>
>
>All the environment variables present in tasks.xml i.e.
>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>
>
>Please let us know how to proceed.
>
>
>Regards,
>Renu
>
>
>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
><mattmann@usc.edu> wrote:
>
>Can you please confirm that all of the bracketed environment
>variables below do in fact exist in the shell that you are
>running ./oodt start in?
>
>E.g., please echo $ENV_VAR below and show their values.
>
>Cheers,
>Chris
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Adjunct Associate Professor, Computer Science Department
>University of Southern California
>Los Angeles, CA 90089 USA
>Email: mattmann@usc.edu
>WWW: http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>-----Original Message-----
>From: Renu Kanakamedala <kanakame@usc.edu>
>Date: Wednesday, November 5, 2014 at 8:18 PM
>To: Chris Mattmann <mattmann@usc.edu>
>Cc: "dev@oodt.apache.org" <dev@oodt.apache.org>, Preethi Ramesh
><pramesh@usc.edu>, Gouthami Kondakindi <kondakin@usc.edu>
>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>parse null Error.
>
>>Please find below tasks.xml file.
>>
>>
>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>><!--
>>  TODO: Add some examples
>>-->
>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>>
>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGET
>>a
>>sk">
>>
>>  <conditions/>
>>
>>  <configuration>
>>     <property name="PGETask_Name" value="FileConcatenator"/>
>>     <property name="PGETask_ConfigFilePath"
>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>>envReplace="true"/>
>>     <property name="PGETask_DumpMetadata" value="true"/>
>>     <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>>envReplace="true" />
>>     <property name="PCS_FileManagerUrl"     value="[FILEMGR_URL]"
>>envReplace="true"/>
>>     <property name="PCS_MetFileExtension" value="met"/>
>>     <property name="PCS_ClientTransferServiceFactory"
>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"
>>/
>>>
>>     <property name="PCS_ActionRepoFile"
>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
>>  </configuration>
>>
>>  <requiredMetFields>
>>     <metfield name="RunID"/>
>>  </requiredMetFields>
>>
>></task>
>></cas:tasks>
>>
>>
>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>><mattmann@usc.edu> wrote:
>>
>>Hi Renu,
>>
>>Can you please show a paste.bin of your tasks.xml?
>>
>>Cheers,
>>Chris
>>
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Chris Mattmann, Ph.D.
>>Adjunct Associate Professor, Computer Science Department
>>University of Southern California
>>Los Angeles, CA 90089 USA
>>Email: mattmann@usc.edu
>>WWW: http://sunset.usc.edu/~mattmann/
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>-----Original Message-----
>>From: Renu Kanakamedala <kanakame@usc.edu>
>>Date: Wednesday, November 5, 2014 at 7:55 PM
>>To: Chris Mattmann <mattmann@usc.edu>
>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
>>null Error.
>>
>>>Hello Professor,
>
>>>
>>>
>>>When i am trying to run file concatenator CAS PGE example using the
>>>following command.
>>>
>>>
>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>>>
>>>
>>>
>>>The message shown after executing this command is "Sending Event
>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>>>stub.
>>>
>>>
>>>Batch Stub:
>>>Caused by:
>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExce
>>>p
>>>t
>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>>value: null
>>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>>at 
>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>>... 2 more
>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>>parse value: null
>>>at
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>o
>>>n
>>>figBuilder.java:87)
>>>at
>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.
>>>j
>>>a
>>>va:230)
>>>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>>... 3 more
>>>Caused by: java.lang.Exception: Failed to parse value: null
>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>>at
>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeC
>>>o
>>>n
>>>figBuilder.java:77)
>>>... 5 more
>>>Caused by: java.lang.NullPointerException
>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>>at java.util.regex.Matcher.reset(Matcher.java:308)
>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>>at
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplace
>>>m
>>>e
>>>nt(PathUtils.java:321)
>>>at
>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUti
>>>l
>>>s
>>>.java:96)
>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>>>... 7 more
>>>
>>>
>>>
>>>
>>>
>>>RES mngr:
>>>
>>>
>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>>>addJob
>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>>>Nov 05, 2014 5:26:30 PM
>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>>>Scheduling for execution
>>>Nov 05, 2014 5:26:30 PM
>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: ProxySelector Request for
>>
>>
>
>
>>>http://localhost:2001/ <http://localhost:2001/>
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: Proxy used: DIRECT
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>writeRequests
>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>getInputStream
>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>text/xml}{Content-Length: 121}
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: ProxySelector Request for
>>>http://localhost:2001/ <http://localhost:2001/>
>
>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>plainConnect
>>>FINEST: Proxy used: DIRECT
>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>>>writeRequests
>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>>>getInputStream
>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>text/xml}{Content-Length: 121}
>>>Nov 05, 2014 5:26:32 PM
>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
>>>batchstub.executeJob returned false
>>>java.lang.Exception: batchstub.executeJob returned false
>>>at
>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatc
>>>h
>>>M
>>>grProxy.java:125)
>>>
>>>
>>>
>>>
>>>
>>>Workflow:
>>>
>>>
>>>Nov 05, 2014 5:26:27 PM
>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>checkTaskRequiredMetadata
>>>INFO: All required metadata fields present for task: [FileConcatenator]
>>>Nov 05, 2014 5:26:28 PM
>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>>submitJob
>>>FINEST: [{job.queueName=high,
>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>>
>>
>
>
>
>
>>>job.name <http://job.name> <http://job.name>
>>><http://job.name>=urn:oodt:FileConcatenator,
>>>job.id <http://job.id> <http://job.id> <http://job.id>=, job.status=,
>>>job.load=2,
>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileC
>>>o
>>>n
>>>catenatorPGETask,
>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/P
>>>G
>>>E
>>>Config.xml,
>>>
>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfe
>>>r
>>>.
>>>LocalDataTransferFactory,
>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xm
>>>l
>>>,
>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>>>PCS_WorkflowManagerUrl=http://localhost:9001,
>>> PCS_FileManagerUrl=http://localhost:9000,
>>>PGETask_Name=FileConcatenator},
>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>>>WorkflowManagerUrl=[http://ubuntu:9001],
>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>>>Nov 05, 2014 5:26:39 PM
>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>stop
>>>WARNING: Attempt to kill current resmgr job:
>>>[2014-11-05T17:26:28.785-08:00]: failed
>>>
>>>
>>>When i searched online i found one of your mail archives where you had
>>>mentioned to change tasks.xml
>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>>>
>>>
>>>
>>>It did not work. Please let us know how to proceed.
>>>
>>>
>>>
>>>Regards,
>>>Renu
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>-- 
>Meghamsh UtkurGraduate Student,University of Southern California
>utkur@usc.edu
>
>(503)881-4791
>
>
>


Mime
View raw message