falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1844) Falcon feed replication leaves behind old files when a feed instance is re-run
Date Wed, 02 Mar 2016 08:29:18 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175260#comment-15175260

Pallavi Rao commented on FALCON-1844:

 Falcon supports deletion of existing files in destination as an optional property. In the
feed definition, the following custom property must be added : 
<property name="removeDeletedFiles" value="true"/>
This property when set to true, deletes files - should there be  any files in the target directory
that were removed from the source directory.

However, this is not default behavior. The fix will set removeDeletedFiles to "true" by default.
User can overwrite it (and set it to false) by specifying the custom property in the feed

> Falcon feed replication leaves behind old files when a feed instance is re-run
> ------------------------------------------------------------------------------
>                 Key: FALCON-1844
>                 URL: https://issues.apache.org/jira/browse/FALCON-1844
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Pallavi Rao
>            Assignee: Pallavi Rao
> When we re-replicate the data from one cluster to another, replication copies and overwrites
old files, but, does not remove extra files in destination (that are not in the source dir
any more). Example, first replication run copied 14 part files, on rerun, the source only
had 13 part files, the 14th file still lingers in the destination.

This message was sent by Atlassian JIRA

View raw message