falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Miklavcic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-391) Add ability to set mapBandwidth
Date Tue, 08 Apr 2014 19:11:15 GMT

    [ https://issues.apache.org/jira/browse/FALCON-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963314#comment-13963314
] 

Michael Miklavcic commented on FALCON-391:
------------------------------------------

I approached this similar to the maxMaps param that passes through to distcp. I referenced
the distcp 2 options here - http://hadoop.apache.org/docs/r1.2.1/distcp2.html#CommandLineOptions.


It looks like the implementation of distcp used by Falcon is custom. Is that an accurate conclusion?
Going deeper into the code I see an inmobi jar named "hadoop-distcp-0.9.jar." In the DistCp
class, createJob appears to set some options, e.g. max maps
{code}
job.getConfiguration().set(DistCpConstants.CONF_LABEL_NUM_MAPS,
            String.valueOf(inputOptions.getMaxMaps()));
{code}
but not mapBandwidth. Is that jar in the purview of the Falcon project? It looks like we might
need to make an additional change there.

> Add ability to set mapBandwidth
> -------------------------------
>
>                 Key: FALCON-391
>                 URL: https://issues.apache.org/jira/browse/FALCON-391
>             Project: Falcon
>          Issue Type: Improvement
>          Components: feed, replication
>    Affects Versions: 0.4, 0.5
>            Reporter: Michael Miklavcic
>            Assignee: Michael Miklavcic
>              Labels: features, patch
>             Fix For: 0.5
>
>         Attachments: FALCON-391_v1.patch
>
>
> Related to https://issues.apache.org/jira/browse/FALCON-342



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message