falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-20) Remove dependency on custom InMobi DistCp
Date Mon, 13 Oct 2014 10:55:34 GMT

    [ https://issues.apache.org/jira/browse/FALCON-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169176#comment-14169176
] 

Shwetha G S commented on FALCON-20:
-----------------------------------

{code}
+            <property>  <!-- DistCp jars -->
+                <name>oozie.action.sharelib.for.java</name>
+                <value>distcp</value>
+            </property>
{code}
Replication will pick distcp jar from oozie sharelib. Is falcon's FeedReplicator guaranteed
to work with any distcp in terms of code signature?

{code}
-        distcpOptions.setMapBandwidthKB(Integer.valueOf(cmd.getOptionValue("mapBandwidthKB")));
-
+        distcpOptions.setMapBandwidth(Integer.valueOf(cmd.getOptionValue("mapBandwidthKB")));
{code}
If latest distcp doesn't support mapBandwidth in KB, we should rename falcon's param as well
accordingly. But this is useful, not sure why it was removed

> Remove dependency on custom InMobi DistCp
> -----------------------------------------
>
>                 Key: FALCON-20
>                 URL: https://issues.apache.org/jira/browse/FALCON-20
>             Project: Falcon
>          Issue Type: Sub-task
>          Components: replication
>    Affects Versions: 0.3
>            Reporter: Venkatesh Seetharam
>            Assignee: Sowmya Ramesh
>             Fix For: 0.6
>
>         Attachments: FALCON-20-C.patch, FALCON-20.patch, FALCON-20.r1.patch, FALCON-20.v2.patch,
cluster01.xml, cluster02.xml, feed.xml
>
>
> Falcon depends on custom DistCp 0.3 since DistCp v2 was part of Hadoop 2.x line. 
> https://issues.apache.org/jira/browse/MAPREDUCE-5081 has back ported DistCp v2 to Hadoop
1.x line and Falcon should instead depend on that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message