oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cayanan, Michael D (388J)" <michael.d.caya...@jpl.nasa.gov>
Subject Re: PGE Question
Date Fri, 02 Mar 2012 23:43:17 GMT
Hi Sheryl,

So, I took your suggestion and put this piece of code together in a post success ingest crawler

    String key = "L0b_Radar_Product_Filename";

    LOG.log(Level.INFO, "Workflow metadata keys: " + metadata.getAllKeys());

    Metadata productNameMet = new Metadata();

    productNameMet.addMetadata(key, product.getName());

      try {

        XmlRpcWorkflowManagerClient wClient = new XmlRpcWorkflowManagerClient(

                new URL(metadata.getMetadata(CoreMetKeys.WORKFLOW_MANAGER_URL)));

        passFlag = wClient.updateMetadataForWorkflow(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID),

        if (passFlag) {

          LOG.log(Level.INFO, "Updated workflow id '" + metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID)
+ "' with metadata: " + productNameMet.getAllKeys());

          Metadata after = wClient.getWorkflowInstanceMetadata(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID));

          LOG.log(Level.INFO, "Metadata after update: " + after.getAllKeys());

I was hoping that this would update my workflow metadata, however it doesn't appear to be
working. I'm seeing this in my logs:

INFO: Workflow metadata keys: [FileLocation, TemporalRange~RangeEndingTime, ProductPathGroup~FileVersionId,
StaticAncillaryFileGroup~L0b_Radar_MetDefFile, TemporalRange~RangeBeginningDate, WorkflowManagerUrl,
TaskId, Filename, InputFileGroup~InputFilePath, StaticAncillaryFileGroup~L0b_Radar_ProductConfig,
TemporalRange~RangeBeginningTime, WorkflowInstId, TemporalRange~RangeEndingDate, ProductType,
JobId, ProductName, StaticAncillaryFileGroup~L0b_Radar_InputConfig, L0a_Radar_Product_Filename,
Mar 2, 2012 3:28:21 PM gov.nasa.smap.spdm.crawl.action.AddProductNameToWorkflowMetAction performAction
INFO: Updated workflow id '6333dd86-64bf-11e1-98c2-afe386bb7e05' with metadata: [L0b_Radar_Product_Filename]
Mar 2, 2012 3:28:22 PM gov.nasa.smap.spdm.crawl.action.AddProductNameToWorkflowMetAction performAction
INFO: Metadata after update: [L0b_Radar_Product_Filename]

Note that even getting the metadata via the workflow client API produces an odd result. It's
only returning the single metadata key that I wanted to add to the current metadata in the
workflow. Not sure why that is. In any case, when I look at the metadata in the task that
follows this PGE, I only see the metadata without the extra metadata key that I wished to

It would seem like I'm doing this correctly, however I'm unsure if I'm missing something or
if this is a bug in the code.


From: Sheryl John <sheryljj@gmail.com<mailto:sheryljj@gmail.com>>
Reply-To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" <user@oodt.apache.org<mailto:user@oodt.apache.org>>
Date: Fri, 2 Mar 2012 12:58:53 -0800
To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" <user@oodt.apache.org<mailto:user@oodt.apache.org>>
Subject: Re: PGE Question

Hi Mike,

You can use any of the Workflow Core met keys inside your PGE.
So, if you want to use the WorkflowInstId, you would specify it like this: [WorkflowInstId]
or [JobId] inside your PGE script.

But there could be other ways to update workflow metadata. You could also have this in you
metFileWriterClass, but that will be before your file is ingested.

On Fri, Mar 2, 2012 at 12:36 PM, Cayanan, Michael D (388J) <michael.d.cayanan@jpl.nasa.gov<mailto:michael.d.cayanan@jpl.nasa.gov>>

In my PGE, I specify my output like the following:


           <dir path="[ProductLocation]" createBeforeExe="true">

              <files regExp="SMAP_L0B_RADAR_SPS.*\.h5" metFileWriterClass="[MetFileWriter]"
args="[ConfigMetoutFile]" />

              <files name="[PGETask_Name]_RunConfig_[ProductionDateTime].xml" metFileWriterClass="[MetFileWriter]"
args="[RunConfigMetoutFile]" />



After ingestion of this output product, I'd like to save the filename in my workflow metadata.
How would I do this?

I tried creating a PostIngestSuccess action, but found out that whatever metadata saving you
want to do on the "crawler" side doesn't get passed back to the workflow metadata.

I suppose I could update the workflow metadata by creating a PostIngestSuccess action and
using the workflow manager client API to do this. However, when I look at this class, I see
the following method,

updateMetadataForWorkflow(String workflowInstId, Metadata metadata)

where it requires the workflow instance ID as a parameter. Is there a way to capture the workflow
instance ID into the workflow metadata so that I can leverage this method to update my metadata?




View raw message