oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cayanan, Michael D (388J)" <michael.d.caya...@jpl.nasa.gov>
Subject Re: PGE Question
Date Sat, 03 Mar 2012 09:18:01 GMT
Thanks Chris. Actually after I sent this message out, I realized that the
easier way is to just store the workflow instance id in the file manager
and querying that for the ingested files (see Brian's reply).

In any case, this is good to know should I ever need to do this in the


On 3/2/12 10:21 PM, "Mattmann, Chris A (388J)"
<chris.a.mattmann@jpl.nasa.gov> wrote:

>Hi Mike,
>On Mar 2, 2012, at 12:36 PM, Cayanan, Michael D (388J) wrote:
>> Hi,
>> In my PGE, I specify my output like the following:
>> 	<output>
>>            <dir path="[ProductLocation]" createBeforeExe="true">
>>               <files regExp="SMAP_L0B_RADAR_SPS.*\.h5"
>>metFileWriterClass="[MetFileWriter]" args="[ConfigMetoutFile]" />
>>               <files
>>metFileWriterClass="[MetFileWriter]" args="[RunConfigMetoutFile]" />
>> 	   </dir>
>> 	</output>
>> After ingestion of this output product, I'd like to save the filename
>>in my workflow metadata. How would I do this?
>You could extend CAS-PGE, and override the protected void
>runIngestCrawler(StdProductCrawler crawler,
>            List<File> crawlDirs) method like so:
>public class SMAPPge extends StdPGETaskInstance{
>   @Override
>protected void runIngestCrawler(StdProductCrawler crawler,
>            List<File> crawlDirs) {
>      super.runIngestCrawler(crawler, crawlDirs);
>      for(IngestStatus status: crawler.getIngestStatus()){
>           this.pgeMetadata.addDyanmicMetadata("IngestedFiles", status.
>      }
>   }
>See this file: http://s.apache.org/ym
>> I tried creating a PostIngestSuccess action, but found out that
>>whatever metadata saving you want to do on the "crawler" side doesn't
>>get passed back to the workflow metadata.
>That's by design. Crawler actions are meant to perform functionality
>"using" a read-only copy of the workflow met. They are *not* intended to
>modify workflow met.
>> I suppose I could update the workflow metadata by creating a
>>PostIngestSuccess action and using the workflow manager client API to do
>>this. However, when I look at this class, I see the following method,
>> updateMetadataForWorkflow(String workflowInstId, Metadata metadata)
>> where it requires the workflow instance ID as a parameter. Is there a
>>way to capture the workflow instance ID into the workflow metadata so
>>that I can leverage this method to update my metadata?
>You've got the workflow instance id. It's given to you by the workflow
>manager in the metadata context for each workflow instance, under the key
>See here: http://s.apache.org/NNs
>and search for "getSharedContext()" for the list of keys that are
>automatically set.
>Chris Mattmann, Ph.D.
>Senior Computer Scientist
>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>Office: 171-266B, Mailstop: 171-246
>Email: chris.a.mattmann@nasa.gov
>WWW:   http://sunset.usc.edu/~mattmann/
>Adjunct Assistant Professor, Computer Science Department
>University of Southern California, Los Angeles, CA 90089 USA

View raw message