falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Bush (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FALCON-1894) HDFS Data replication cannot be initiated independent of Oozie server location
Date Mon, 11 Apr 2016 15:09:25 GMT
Alex Bush created FALCON-1894:
---------------------------------

             Summary: HDFS Data replication cannot be initiated independent of Oozie server
location
                 Key: FALCON-1894
                 URL: https://issues.apache.org/jira/browse/FALCON-1894
             Project: Falcon
          Issue Type: Bug
          Components: general
    Affects Versions: trunk
            Reporter: Alex Bush
            Priority: Minor


The HDFS mirroring scripts allow replication between two clusters.
Currently, even though the UI allows the replication in any direction between clusters independent
of which cluster the Falcon and Oozie servers belong to this is not observed and the source
cluster is always the cluster with Oozie/Falcon server.

Steps to reproduce:
1) Define both clusters in Falcon server on cluster 2
2) Set up HDFS mirroring in Falcon server on cluster 2 from Cluster 1 to Cluster 2 and set
to run on Oozie server of Cluster 2

Result:
Falcon will replicate data from Cluster 2 to Cluster 2

Cause:
In hdfs-replication-workflow.xml, the source dir should be defined like the target dir by
including clusterfs:
https://github.com/apache/falcon/blob/master/addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-workflow.xml#L63

<arg>${drSourceDir}</arg>
should be
<arg>${drSourceClusterFS}${drSourceDir}</arg>
 
like

https://github.com/apache/falcon/blob/master/addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-workflow.xml#L65



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message