falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-481) Simplify process parent workflow
Date Thu, 19 Jun 2014 07:35:24 GMT

    [ https://issues.apache.org/jira/browse/FALCON-481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037072#comment-14037072

Shwetha G S commented on FALCON-481:

Why spit actions that are not necessary, like pig engine should only result in a pig action
and remove the decision node in oozie.
Yes, thats what I am saying. Pig process can have a simplified workflow with just - [recordsize],
pig action and post-processing. We can get rid of all decision nodes

Also may make sense to convert the record size to falcon pre processing and have things in
there for extensibility, may be another jira
Yes, when we have some logic in pre-processing, we can add it. We can skip it for now and
avoid switch case

> Simplify process parent workflow
> --------------------------------
>                 Key: FALCON-481
>                 URL: https://issues.apache.org/jira/browse/FALCON-481
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Shwetha G S
> Currently, process parent workflow is one big workflow with switch cases to handle recordSize,
pig workflow, oozie workflow and hive workflow. Each action in the parent workflow creates
a row in WF_ACTIONS table in oozie and can bloat up the size. Instead create a simplified
workflow without switch cases that works for the usecase

This message was sent by Atlassian JIRA

View raw message