oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: PGE Question
Date Sat, 03 Mar 2012 18:53:49 GMT
Anytime, man!

Cheers,
Chris

On Mar 3, 2012, at 1:18 AM, Cayanan, Michael D (388J) wrote:

> Thanks Chris. Actually after I sent this message out, I realized that the
> easier way is to just store the workflow instance id in the file manager
> and querying that for the ingested files (see Brian's reply).
> 
> In any case, this is good to know should I ever need to do this in the
> future!
> 
> Cheers,
> Mike
> 
> On 3/2/12 10:21 PM, "Mattmann, Chris A (388J)"
> <chris.a.mattmann@jpl.nasa.gov> wrote:
> 
>> Hi Mike,
>> 
>> On Mar 2, 2012, at 12:36 PM, Cayanan, Michael D (388J) wrote:
>> 
>>> Hi,
>>> 
>>> In my PGE, I specify my output like the following:
>>> 
>>> 	<output>
>>>           <dir path="[ProductLocation]" createBeforeExe="true">
>>>              <files regExp="SMAP_L0B_RADAR_SPS.*\.h5"
>>> metFileWriterClass="[MetFileWriter]" args="[ConfigMetoutFile]" />
>>>              <files
>>> name="[PGETask_Name]_RunConfig_[ProductionDateTime].xml"
>>> metFileWriterClass="[MetFileWriter]" args="[RunConfigMetoutFile]" />
>>> 	   </dir>
>>> 	</output>
>>> 
>>> After ingestion of this output product, I'd like to save the filename
>>> in my workflow metadata. How would I do this?
>> 
>> You could extend CAS-PGE, and override the protected void
>> runIngestCrawler(StdProductCrawler crawler,
>>           List<File> crawlDirs) method like so:
>> 
>> public class SMAPPge extends StdPGETaskInstance{
>> 
>>  @Override
>> protected void runIngestCrawler(StdProductCrawler crawler,
>>           List<File> crawlDirs) {
>>     super.runIngestCrawler(crawler, crawlDirs);
>> 
>>     for(IngestStatus status: crawler.getIngestStatus()){
>>          this.pgeMetadata.addDyanmicMetadata("IngestedFiles", status.
>> getProduct().getName());
>>     }
>> 
>>  }
>> 
>> See this file: http://s.apache.org/ym
>> 
>>> 
>>> I tried creating a PostIngestSuccess action, but found out that
>>> whatever metadata saving you want to do on the "crawler" side doesn't
>>> get passed back to the workflow metadata.
>> 
>> That's by design. Crawler actions are meant to perform functionality
>> "using" a read-only copy of the workflow met. They are *not* intended to
>> modify workflow met.
>> 
>>> 
>>> I suppose I could update the workflow metadata by creating a
>>> PostIngestSuccess action and using the workflow manager client API to do
>>> this. However, when I look at this class, I see the following method,
>>> 
>>> 
>>> updateMetadataForWorkflow(String workflowInstId, Metadata metadata)
>>> 
>>> 
>>> where it requires the workflow instance ID as a parameter. Is there a
>>> way to capture the workflow instance ID into the workflow metadata so
>>> that I can leverage this method to update my metadata?
>> 
>> You've got the workflow instance id. It's given to you by the workflow
>> manager in the metadata context for each workflow instance, under the key
>> name:
>> 
>> "WorkflowInstId"
>> 
>> See here: http://s.apache.org/NNs
>> 
>> and search for "getSharedContext()" for the list of keys that are
>> automatically set.
>> 
>> Cheers,
>> Chris
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 
> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message