falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FALCON-206) Process update for wf changes
Date Tue, 24 Dec 2013 07:44:50 GMT

     [ https://issues.apache.org/jira/browse/FALCON-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Shwetha G S updated FALCON-206:

    Attachment: FALCON-206.patch

At process schedule, process mapper creates a checksum file that contains the list of user
wf/lib files and their checksum. The process mapper also copies the user wf and lib to staging
directory(where parent wf and coord xml are stored). The parent workflow references the user
workflow copied to staging dir. 
Process update updates in oozie only if entity is updated or user wf/lib is updated.

With this change, when user wf/lib is updated, the currently running instances will not fail.
This will also make adding new input to process easier (which includes updating process and

Limitations: user oozie workflow should use only relative paths for libs/pig scripts/hive

> Process update for wf changes
> -----------------------------
>                 Key: FALCON-206
>                 URL: https://issues.apache.org/jira/browse/FALCON-206
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Shwetha G S
>            Assignee: Shwetha G S
>         Attachments: FALCON-206.patch
> Updating a process for adding another input requires updating process definition(add
input) and oozie wf(to handle new input). When oozie wf is updated, all instances take the
new definition. Re-running an old instance also uses the  new wf definition, but the wf will
fail as the new input is not defined for old instances.
> To solve this, the wf and the lib should be copied to a staging directory with every
process updatte

This message was sent by Atlassian JIRA

View raw message