falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-93) Replication to handle hive table replication
Date Thu, 10 Oct 2013 23:20:41 GMT

    [ https://issues.apache.org/jira/browse/FALCON-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792091#comment-13792091
] 

Venkatesh Seetharam commented on FALCON-93:
-------------------------------------------

I also wanted your opinion on cleaning the staging area. There are 2 options:

* Use org.apache.falcon.cleanup.FeedCleanupHandler in org.apache.falcon.service.LogCleanupService
I have implemented this one. Might help to retry using -update option in DistCp. 
* Delete these paths post export as part of an fs action in oozie itself. 
{code}
    <action name="table-import">
         ......
        <ok to="cleanup-staging-area"/>
        <error to="fail "/>
    </action>
    <action name="cleanup-staging-area">
        <fs>
            <delete path="${sourceStagingDir}/${nominalTime}" />
            <delete path="${targetStagingDir}/${nominalTime}" />
        </fs>
        <ok to="succeeded-post-processing"/>
        <error to="failed-post-processing"/>
    </action>
{code}

> Replication to handle hive table replication
> --------------------------------------------
>
>                 Key: FALCON-93
>                 URL: https://issues.apache.org/jira/browse/FALCON-93
>             Project: Falcon
>          Issue Type: Sub-task
>    Affects Versions: 0.3
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>         Attachments: FALCON-93.patch, FALCON-93-r1.patch
>
>
> Data and metadata to be replicated atomically.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message