falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sowmya Ramesh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FALCON-2028) HDFS extension: Validate and append/remove the scheme://authority for the paths
Date Tue, 14 Jun 2016 20:54:30 GMT
Sowmya Ramesh created FALCON-2028:
-------------------------------------

             Summary: HDFS extension: Validate and append/remove the scheme://authority for
the paths
                 Key: FALCON-2028
                 URL: https://issues.apache.org/jira/browse/FALCON-2028
             Project: Falcon
          Issue Type: Bug
            Reporter: Sowmya Ramesh
            Assignee: Sowmya Ramesh
             Fix For: trunk


In hdfs-mirroring-workflow.xml target EP is added only for target dir.

{noformat}
<arg>-sourcePaths</arg>
<arg>${sourceDir}</arg>
<arg>-targetPath</arg>
<arg>${targetClusterFS}${targetDir}</arg>
{noformat}

in the req, if sourceDir is not sent as fully qualified paths then extension should generate
fully qualified paths before submitting the process. For targetDir if passed it should be
removed.

e.g.

{noformat}
sourceDir=/user/ambari-qaqa/dr/test/primaryCluster/input1, /user/ambari-qaqa/dr/test/primaryCluster/input2

After processing:
sourceDir=hdfs://240.0.0.10:8020/user/ambari-qaqa/dr/test/primaryCluster/input1, hdfs://240.0.0.10:8020/user/ambari-qaqa/dr/test/primaryCluster/input2
{noformat}

Also, if user has already specified the fully qualified path then it should not be added for
sourceDir but it should be removed for targetDir

{noformat}
sourceDir=hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input1, hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input2

After processing:
sourceDir=hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input1, hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input2
================================================================

targetDir=hdfs://240.0.0.11:8020/user/ambari-qa/dr/test/backupCluster/outputs

After processing:
targetDir=/user/ambari-qa/dr/test/backupCluster/outputs

{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message