falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1835) Falcon should do coord rerun rather than workflow rerun to ensure concurrency
Date Tue, 23 Feb 2016 10:43:18 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158701#comment-15158701
] 

ASF GitHub Bot commented on FALCON-1835:
----------------------------------------

GitHub user pallavi-rao opened a pull request:

    https://github.com/apache/falcon/pull/48

    FALCON-1835 Falcon should do coord rerun rather than workflow rerun ...

    …to ensure concurrency
    
    Patch contains the following changes:
    1. Upgrade to Oozie client 4.2 (from 4.1).
    2. Remove workflow rerun completely and switch to coord rerun.
    3. Introduced parentId in WorkflowExecutionContext in order to store the coord action
of a given workflow. This will be used during retry and late data rerun.
    4. In cases where parentId cannot be populated (notification from Post processing action
will not contain this), rerun method queries Oozie to retrieve the parent id.
    5. The rerun method now returns the actual status after rerun, rather than hardcoded “RUNNING”.
    6. In 4.2, Oozie does a Yarn job kill to kill any lingering executions of previous run
of a workflow. It uses RMClientProxy to do the same. Since in local mode, RPC cannot happen,
had to introduce LocalFalconRPCClientFactory to prevent RPC in local mode.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/pallavi-rao/falcon 1835

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/falcon/pull/48.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #48
    
----
commit 43df66d56dbe4a674e6a8499b93cf7a14c076f83
Author: Pallavi Rao <pallavi.rao@inmobi.com>
Date:   2016-02-23T10:38:51Z

    FALCON-1835 Falcon should do coord rerun rather than workflow rerun to ensure concurrency

----


> Falcon should do coord rerun rather than workflow rerun to ensure concurrency
> -----------------------------------------------------------------------------
>
>                 Key: FALCON-1835
>                 URL: https://issues.apache.org/jira/browse/FALCON-1835
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Pallavi Rao
>            Assignee: Pallavi Rao
>              Labels: backward-incompatible
>
> As of today, falcon does a workflow rerun, which can cause parallel instances to run.
> 1) Rerun via falcon retry.
> 2) Coord rerun via direct oozie command line.
> Falcon needs to move to coord rerun to rerun for same workflow which is already present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message