oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Alan Mattmann <mattm...@usc.edu>
Subject Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.
Date Thu, 06 Nov 2014 06:01:03 GMT
Can you add lines:

-Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \
-Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \

Starting at line 92 in your batch_stub script, restart
the batch stub and see if that fixes it?

Cheers,
Chris



++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Adjunct Associate Professor, Computer Science Department
University of Southern California
Los Angeles, CA 90089 USA
Email: mattmann@usc.edu
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Renu Kanakamedala <kanakame@usc.edu>
Date: Wednesday, November 5, 2014 at 10:51 PM
To: Chris Mattmann <mattmann@usc.edu>
Cc: Meghamsh Utkur <utkur@usc.edu>, "dev@oodt.apache.org"
<dev@oodt.apache.org>, Preethi Ramesh <pramesh@usc.edu>, Gouthami
Kondakindi <kondakin@usc.edu>
Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
parse null Error.

>wmgr script 
>
>http://pastebin.com/zE7PWnXL
>
>
>resmgr batch_stub
>
>
>http://pastebin.com/gPUvbE9e
>
>
>- Renu
>
>
>
>
>
>On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann
><mattmann@usc.edu> wrote:
>
>Great. One last question.
>
>Can you show me the contents (paste.bin) of your wmgr
>script and your batch_stub script?
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>Chris Mattmann, Ph.D.
>Adjunct Associate Professor, Computer Science Department
>University of Southern California
>Los Angeles, CA 90089 USA
>Email: mattmann@usc.edu
>WWW: http://sunset.usc.edu/~mattmann/
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>-----Original Message-----
>From: Renu Kanakamedala <kanakame@usc.edu>
>Date: Wednesday, November 5, 2014 at 10:20 PM
>To: Chris Mattmann <mattmann@usc.edu>
>Cc: Meghamsh Utkur <utkur@usc.edu>, "dev@oodt.apache.org"
><dev@oodt.apache.org>, Preethi Ramesh <pramesh@usc.edu>, Gouthami
>Kondakindi <kondakin@usc.edu>
>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>parse null Error.
>
>>renu@ubuntu:~$ echo $WORKFLOW_HOME
>>/home/renu/oodt/workflow/
>>renu@ubuntu:~$ echo $FILEMGR_HOME
>>/home/renu/oodt/filemgr
>>renu@ubuntu:~$ echo $WORKFLOW_URL
>>http://localhost:9001
>>renu@ubuntu:~$ echo $FILEMGR_URL
>>http://localhost:9000
>>renu@ubuntu:~$ echo $PGE_ROOT
>>/home/renu/oodt/pge/
>>
>>renu@ubuntu:~$ echo $CRAWLER_HOME
>>/home/renu/oodt/crawler/
>>
>>
>>
>>
>>
>>
>>
>>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann
>><mattmann@usc.edu> wrote:
>>
>>Right, which is why I asked to specifically echo the value
>>of all environment variables so we would be able to see the
>>values. Can Renu please show the echo values?
>>
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>Chris Mattmann, Ph.D.
>>Adjunct Associate Professor, Computer Science Department
>>University of Southern California
>>Los Angeles, CA 90089 USA
>>Email: mattmann@usc.edu
>>WWW: http://sunset.usc.edu/~mattmann/
>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>>-----Original Message-----
>>From: Meghamsh Utkur <utkur@usc.edu>
>>Date: Wednesday, November 5, 2014 at 9:35 PM
>>To: Renu Kanakamedala <kanakame@usc.edu>
>>Cc: Chris Mattmann <mattmann@usc.edu>, "dev@oodt.apache.org"
>><dev@oodt.apache.org>, Preethi Ramesh <pramesh@usc.edu>, Gouthami
>>Kondakindi <kondakin@usc.edu>
>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>parse null Error.
>>
>>>Professor,
>>>
>>>
>>>To add to what Renu has said, I went through the source code and I
>>>suspect that the value here in this method is passed as null.
>>>
>>>XmlFilePgeConfigBuilder.java
>>>
>>>1. private String fillIn(String value,Metadata inputMetadata,boolean
>>>envReplaceRecur)
>>>2. throwsException {
>>>3. try{
>>>4. while((value =PathUtils.doDynamicReplacement(value,
>>>inputMetadata)).contains("[")&& envReplaceRecur);
>>>5. if(value.toUpperCase()
>>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
>>>7. value
>>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new
>>>URL(inputMetadata
>>>8.
>>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.
>>>p
>>>a
>>>rseSqlQueryMethod(value)));
>>>9. return value;
>>>10. }catch(Exception e){
>>>11. thrownew Exception("Failed to parse value: "+ value, e);
>>>12. }
>>>13. }
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala
>>><kanakame@usc.edu> wrote:
>>>
>>>Hello Professor,
>>>
>>>
>>>All the environment variables present in tasks.xml i.e.
>>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>>>
>>>
>>>Please let us know how to proceed.
>>>
>>>
>>>Regards,
>>>Renu
>>>
>>>
>>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann
>>><mattmann@usc.edu> wrote:
>>>
>>>Can you please confirm that all of the bracketed environment
>>>variables below do in fact exist in the shell that you are
>>>running ./oodt start in?
>>>
>>>E.g., please echo $ENV_VAR below and show their values.
>>>
>>>Cheers,
>>>Chris
>>>
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>Chris Mattmann, Ph.D.
>>>Adjunct Associate Professor, Computer Science Department
>>>University of Southern California
>>>Los Angeles, CA 90089 USA
>>>Email: mattmann@usc.edu
>>>WWW: http://sunset.usc.edu/~mattmann/
>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: Renu Kanakamedala <kanakame@usc.edu>
>>>Date: Wednesday, November 5, 2014 at 8:18 PM
>>>To: Chris Mattmann <mattmann@usc.edu>
>>>Cc: "dev@oodt.apache.org" <dev@oodt.apache.org>, Preethi Ramesh
>>><pramesh@usc.edu>, Gouthami Kondakindi <kondakin@usc.edu>
>>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>parse null Error.
>>>
>>>>Please find below tasks.xml file.
>>>>
>>>>
>>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>>>><!--
>>>>  TODO: Add some examples
>>>>-->
>>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>>>>
>>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPG
>>>>E
>>>>T
>>>>a
>>>>sk">
>>>>
>>>>  <conditions/>
>>>>
>>>>  <configuration>
>>>>     <property name="PGETask_Name" value="FileConcatenator"/>
>>>>     <property name="PGETask_ConfigFilePath"
>>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>>>>envReplace="true"/>
>>>>     <property name="PGETask_DumpMetadata" value="true"/>
>>>>     <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>>>>envReplace="true" />
>>>>     <property name="PCS_FileManagerUrl"     value="[FILEMGR_URL]"
>>>>envReplace="true"/>
>>>>     <property name="PCS_MetFileExtension" value="met"/>
>>>>     <property name="PCS_ClientTransferServiceFactory"
>>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactor
>>>>y
>>>>"
>>>>/
>>>>>
>>>>     <property name="PCS_ActionRepoFile"
>>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml"
>>>>envReplace="true"/>
>>>>  </configuration>
>>>>
>>>>  <requiredMetFields>
>>>>     <metfield name="RunID"/>
>>>>  </requiredMetFields>
>>>>
>>>></task>
>>>></cas:tasks>
>>>>
>>>>
>>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>>>><mattmann@usc.edu> wrote:
>>>>
>>>>Hi Renu,
>>>>
>>>>Can you please show a paste.bin of your tasks.xml?
>>>>
>>>>Cheers,
>>>>Chris
>>>>
>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>Chris Mattmann, Ph.D.
>>>>Adjunct Associate Professor, Computer Science Department
>>>>University of Southern California
>>>>Los Angeles, CA 90089 USA
>>>>Email: mattmann@usc.edu
>>>>WWW: http://sunset.usc.edu/~mattmann/
>>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>
>>>>
>>>>
>>>>
>>>>-----Original Message-----
>>>>From: Renu Kanakamedala <kanakame@usc.edu>
>>>>Date: Wednesday, November 5, 2014 at 7:55 PM
>>>>To: Chris Mattmann <mattmann@usc.edu>
>>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>>>>parse
>>>>null Error.
>>>>
>>>>>Hello Professor,
>>>
>>>>>
>>>>>
>>>>>When i am trying to run file concatenator CAS PGE example using the
>>>>>following command.
>>>>>
>>>>>
>>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>>>>>
>>>>>
>>>>>
>>>>>The message shown after executing this command is "Sending Event
>>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and
>>>>>batch
>>>>>stub.
>>>>>
>>>>>
>>>>>Batch Stub:
>>>>>Caused by:
>>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceEx
>>>>>c
>>>>>e
>>>>>p
>>>>>t
>>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>>>>>value: null
>>>>>at
>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>>>>>at
>>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>>>>>... 2 more
>>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>>>>>parse value: null
>>>>>at
>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>>>>>e
>>>>>C
>>>>>o
>>>>>n
>>>>>figBuilder.java:87)
>>>>>at
>>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc
>>>>>e
>>>>>.
>>>>>j
>>>>>a
>>>>>va:230)
>>>>>at
>>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>>>>>... 3 more
>>>>>Caused by: java.lang.Exception: Failed to parse value: null
>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>>>>>at
>>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg
>>>>>e
>>>>>C
>>>>>o
>>>>>n
>>>>>figBuilder.java:77)
>>>>>... 5 more
>>>>>Caused by: java.lang.NullPointerException
>>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>>>>>at java.util.regex.Matcher.reset(Matcher.java:308)
>>>>>at java.util.regex.Matcher.<init>(Matcher.java:228)
>>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>>>>>at
>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla
>>>>>c
>>>>>e
>>>>>m
>>>>>e
>>>>>nt(PathUtils.java:321)
>>>>>at
>>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU
>>>>>t
>>>>>i
>>>>>l
>>>>>s
>>>>>.java:96)
>>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>>>>>... 7 more
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>RES mngr:
>>>>>
>>>>>
>>>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>>>>>addJob
>>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>>>>>Nov 05, 2014 5:26:30 PM
>>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>>>>>Scheduling for execution
>>>>>Nov 05, 2014 5:26:30 PM
>>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: ProxySelector Request for
>>>>
>>>>
>>>
>>>
>>>>>http://localhost:2001/ <http://localhost:2001/>
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: Proxy used: DIRECT
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>writeRequests
>>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>>> q=.2}{Connection: keep-alive}{Content-Length: 105}
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>getInputStream
>>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>>text/xml}{Content-Length: 121}
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: ProxySelector Request for
>>>>>http://localhost:2001/ <http://localhost:2001/>
>>>
>>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>plainConnect
>>>>>FINEST: Proxy used: DIRECT
>>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>writeRequests
>>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>>>>>text/html, image/gif, image/jpeg, *; q=.2, */*;
>>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>>>>>getInputStream
>>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>>>>>text/xml}{Content-Length: 121}
>>>>>Nov 05, 2014 5:26:32 PM
>>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>>>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00'
>>>>>:
>>>>>batchstub.executeJob returned false
>>>>>java.lang.Exception: batchstub.executeJob returned false
>>>>>at
>>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBa
>>>>>t
>>>>>c
>>>>>h
>>>>>M
>>>>>grProxy.java:125)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>Workflow:
>>>>>
>>>>>
>>>>>Nov 05, 2014 5:26:27 PM
>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>>checkTaskRequiredMetadata
>>>>>INFO: All required metadata fields present for task:
>>>>>[FileConcatenator]
>>>>>Nov 05, 2014 5:26:28 PM
>>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>>>>>submitJob
>>>>>FINEST: [{job.queueName=high,
>>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>>>job.name <http://job.name> <http://job.name> <http://job.name>
>>>>><http://job.name>
>>>>><http://job.name>=urn:oodt:FileConcatenator,
>>
>>
>>>>>job.id <http://job.id> <http://job.id> <http://job.id>
<http://job.id>
>>>>><http://job.id>=, job.status=,
>>>>>job.load=2,
>>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.Fil
>>>>>e
>>>>>C
>>>>>o
>>>>>n
>>>>>catenatorPGETask,
>>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs
>>>>>/
>>>>>P
>>>>>G
>>>>>E
>>>>>Config.xml,
>>>>>
>>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatrans
>>>>>f
>>>>>e
>>>>>r
>>>>>.
>>>>>LocalDataTransferFactory,
>>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.
>>>>>x
>>>>>m
>>>>>l
>>>>>,
>>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>>>>>PCS_WorkflowManagerUrl=http://localhost:9001,
>>>>> PCS_FileManagerUrl=http://localhost:9000,
>>>>>PGETask_Name=FileConcatenator},
>>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>>>>>WorkflowManagerUrl=[http://ubuntu:9001],
>>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>>>>> ProcessingNode=[ubuntu], RunID=[testNumber1],
>>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>>>>>Nov 05, 2014 5:26:39 PM
>>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>>>>>stop
>>>>>WARNING: Attempt to kill current resmgr job:
>>>>>[2014-11-05T17:26:28.785-08:00]: failed
>>>>>
>>>>>
>>>>>When i searched online i found one of your mail archives where you had
>>>>>mentioned to change tasks.xml
>>>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html
>>>>>
>>>>>
>>>>>
>>>>>It did not work. Please let us know how to proceed.
>>>>>
>>>>>
>>>>>
>>>>>Regards,
>>>>>Renu
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>--
>>
>>
>>>Meghamsh UtkurGraduate Student,University of Southern California
>>>utkur@usc.edu
>>>
>
>
>>>(503)881-4791 <tel:%28503%29881-4791> <tel:%28503%29881-4791>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
>


Mime
View raw message