Return-Path: X-Original-To: apmail-oodt-dev-archive@www.apache.org Delivered-To: apmail-oodt-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 60F739B9C for ; Sat, 3 Mar 2012 18:54:21 +0000 (UTC) Received: (qmail 75195 invoked by uid 500); 3 Mar 2012 18:54:21 -0000 Delivered-To: apmail-oodt-dev-archive@oodt.apache.org Received: (qmail 75170 invoked by uid 500); 3 Mar 2012 18:54:21 -0000 Mailing-List: contact user-help@oodt.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@oodt.apache.org Delivered-To: mailing list user@oodt.apache.org Received: (qmail 75162 invoked by uid 99); 3 Mar 2012 18:54:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Mar 2012 18:54:21 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [128.149.139.105] (HELO mail.jpl.nasa.gov) (128.149.139.105) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Mar 2012 18:54:13 +0000 Received: from mail.jpl.nasa.gov (altvirehtstap02.jpl.nasa.gov [128.149.137.73]) by smtp.jpl.nasa.gov (Switch-3.4.3/Switch-3.4.3) with ESMTP id q23Iro6k001765 (using TLSv1/SSLv3 with cipher RC4-MD5 (128 bits) verified NO) for ; Sat, 3 Mar 2012 10:53:50 -0800 Received: from ALTPHYEMBEVSP20.RES.AD.JPL ([128.149.137.83]) by ALTVIREHTSTAP02.RES.AD.JPL ([128.149.137.73]) with mapi; Sat, 3 Mar 2012 10:53:49 -0800 From: "Mattmann, Chris A (388J)" To: "user@oodt.apache.org" Date: Sat, 3 Mar 2012 10:53:49 -0800 Subject: Re: PGE Question Thread-Topic: PGE Question Thread-Index: Acz5bvgwiv4opaoESXGMhoPc7/4UYQ== Message-ID: <7064001B-302A-471A-AEC8-CA59470E120C@jpl.nasa.gov> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Source-IP: altvirehtstap02.jpl.nasa.gov [128.149.137.73] X-Source-Sender: chris.a.mattmann@jpl.nasa.gov X-AUTH: Authorized X-Virus-Checked: Checked by ClamAV on apache.org Anytime, man! Cheers, Chris On Mar 3, 2012, at 1:18 AM, Cayanan, Michael D (388J) wrote: > Thanks Chris. Actually after I sent this message out, I realized that the > easier way is to just store the workflow instance id in the file manager > and querying that for the ingested files (see Brian's reply). >=20 > In any case, this is good to know should I ever need to do this in the > future! >=20 > Cheers, > Mike >=20 > On 3/2/12 10:21 PM, "Mattmann, Chris A (388J)" > wrote: >=20 >> Hi Mike, >>=20 >> On Mar 2, 2012, at 12:36 PM, Cayanan, Michael D (388J) wrote: >>=20 >>> Hi, >>>=20 >>> In my PGE, I specify my output like the following: >>>=20 >>> >>> >>> >> metFileWriterClass=3D"[MetFileWriter]" args=3D"[ConfigMetoutFile]" /> >>> >> name=3D"[PGETask_Name]_RunConfig_[ProductionDateTime].xml" >>> metFileWriterClass=3D"[MetFileWriter]" args=3D"[RunConfigMetoutFile]" /= > >>> >>> >>>=20 >>> After ingestion of this output product, I'd like to save the filename >>> in my workflow metadata. How would I do this? >>=20 >> You could extend CAS-PGE, and override the protected void >> runIngestCrawler(StdProductCrawler crawler, >> List crawlDirs) method like so: >>=20 >> public class SMAPPge extends StdPGETaskInstance{ >>=20 >> @Override >> protected void runIngestCrawler(StdProductCrawler crawler, >> List crawlDirs) { >> super.runIngestCrawler(crawler, crawlDirs); >>=20 >> for(IngestStatus status: crawler.getIngestStatus()){ >> this.pgeMetadata.addDyanmicMetadata("IngestedFiles", status. >> getProduct().getName()); >> } >>=20 >> } >>=20 >> See this file: http://s.apache.org/ym >>=20 >>>=20 >>> I tried creating a PostIngestSuccess action, but found out that >>> whatever metadata saving you want to do on the "crawler" side doesn't >>> get passed back to the workflow metadata. >>=20 >> That's by design. Crawler actions are meant to perform functionality >> "using" a read-only copy of the workflow met. They are *not* intended to >> modify workflow met. >>=20 >>>=20 >>> I suppose I could update the workflow metadata by creating a >>> PostIngestSuccess action and using the workflow manager client API to d= o >>> this. However, when I look at this class, I see the following method, >>>=20 >>>=20 >>> updateMetadataForWorkflow(String workflowInstId, Metadata metadata) >>>=20 >>>=20 >>> where it requires the workflow instance ID as a parameter. Is there a >>> way to capture the workflow instance ID into the workflow metadata so >>> that I can leverage this method to update my metadata? >>=20 >> You've got the workflow instance id. It's given to you by the workflow >> manager in the metadata context for each workflow instance, under the ke= y >> name: >>=20 >> "WorkflowInstId" >>=20 >> See here: http://s.apache.org/NNs >>=20 >> and search for "getSharedContext()" for the list of keys that are >> automatically set. >>=20 >> Cheers, >> Chris >>=20 >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Chris Mattmann, Ph.D. >> Senior Computer Scientist >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> Office: 171-266B, Mailstop: 171-246 >> Email: chris.a.mattmann@nasa.gov >> WWW: http://sunset.usc.edu/~mattmann/ >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Adjunct Assistant Professor, Computer Science Department >> University of Southern California, Los Angeles, CA 90089 USA >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>=20 >=20 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattmann@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++