falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-481) Simplify process parent workflow
Date Thu, 19 Jun 2014 07:35:24 GMT

    [ https://issues.apache.org/jira/browse/FALCON-481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037072#comment-14037072
] 

Shwetha G S commented on FALCON-481:
------------------------------------

{quote}
Why spit actions that are not necessary, like pig engine should only result in a pig action
and remove the decision node in oozie.
{quote}
Yes, thats what I am saying. Pig process can have a simplified workflow with just - [recordsize],
pig action and post-processing. We can get rid of all decision nodes

{quote}
Also may make sense to convert the record size to falcon pre processing and have things in
there for extensibility, may be another jira
{quote}
Yes, when we have some logic in pre-processing, we can add it. We can skip it for now and
avoid switch case

> Simplify process parent workflow
> --------------------------------
>
>                 Key: FALCON-481
>                 URL: https://issues.apache.org/jira/browse/FALCON-481
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Shwetha G S
>
> Currently, process parent workflow is one big workflow with switch cases to handle recordSize,
pig workflow, oozie workflow and hive workflow. Each action in the parent workflow creates
a row in WF_ACTIONS table in oozie and can bloat up the size. Instead create a simplified
workflow without switch cases that works for the usecase



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message