falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1405) Oozie workflow should not fail if post-processing fails
Date Wed, 19 Aug 2015 07:21:46 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702629#comment-14702629

Pallavi Rao commented on FALCON-1405:

Post-processing is a Java action and can fail due to various reasons,  such as, 
1. Interim issues with Hadoop/YARN.
2. Post processing also does some log movement. This might fail because of HDFS quota issue.
3. The JMS destination may be temporarily down.

If it is a code issue, we can fix. If it is an temporary infra issue, can't do much there.
But, the point is, if the user workflow goes through fine and the post-processing alone fails,
we should still treat it as SUCCESS, as the user workflow has completed.

> Oozie workflow should not fail if post-processing fails
> -------------------------------------------------------
>                 Key: FALCON-1405
>                 URL: https://issues.apache.org/jira/browse/FALCON-1405
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Pallavi Rao
>            Assignee: Pallavi Rao
> Currently, even if the user workflow succeeds and post-processing fails, the workflow
is marked as failed.
> IMO, the post-processing should be an optional step. We should wire workflow to succeed
even if the post-processing fails. Especially, after FALCON-1231, we won't even depend on
post-processing for completion notification.

This message was sent by Atlassian JIRA

View raw message