falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Sundarrajan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-30) Enable embedding pig scripts directly in a process
Date Sat, 13 Jul 2013 06:49:48 GMT

    [ https://issues.apache.org/jira/browse/FALCON-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13707673#comment-13707673
] 

Srikanth Sundarrajan commented on FALCON-30:
--------------------------------------------

Incremental processing is less common and the proposal is to handle that through a custom
workflow. The current patch seems to make it easy for the most common use case. In future
with support for hcatalog tables, rmf is inappropriate, as that would leave a dangling partition
in the meta store.

If conf can be read and accessed in pig script just like params, then we dont need to duplicate
it, else it will be useful to add both conf & properties to params as [~shwethags] suggests.
                
> Enable embedding pig scripts directly in a process
> --------------------------------------------------
>
>                 Key: FALCON-30
>                 URL: https://issues.apache.org/jira/browse/FALCON-30
>             Project: Falcon
>          Issue Type: Improvement
>          Components: process
>    Affects Versions: 0.3
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>              Labels: process
>             Fix For: 0.3
>
>         Attachments: FALCON-30.patch, FALCON-30.r2.patch, FALCON-30.r3.patch, FALCON-30.rev.patch
>
>
> Falcon allows users to express processing as a oozie workflow. This will enable users
to embed pig or hive scripts with out having to express them in a oozie workflow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message