Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <742219485.132661268041528041.JavaMail.jira@brutus.apache.org>
Date: Mon, 8 Mar 2010 09:45:28 +0000 (UTC)
From: "Ravi Gummadi (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-653) distcp can support bandwidth
 limiting
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842604#action_12842604 ] 

Ravi Gummadi commented on MAPREDUCE-653:
----------------------------------------

Yes. But we need to make sure that a particular distcp-Job's task on a node is not hogging the whole bandwidth available for the node(just to be fair to the other job's tasks running on this node).
 Also at the time of launching the distcp job, we will not have enough information on the number of tasks of this job that will run paralelly on the cluster at different points of time(because (a) at some point few tasks of this distcp job could be running --- allowing tasks of this distcp job to use high bandwidth per task and (b) at some other time, different number of tasks of this distcp job running --- allowing tasks of this distcp job to use low bandwidth per task).
So if sleeping by map task of distcp job looks bad, then distcp needs to find out a way of identifying the number of map tasks to be run(actually, number of map tasks that can run paralelly on the cluster at any point of time ?) so that it can set the bandwidth usage per task properly with lower sleep time.

> distcp can support bandwidth limiting
> -------------------------------------
>
>                 Key: MAPREDUCE-653
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-653
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: distcp
>            Reporter: Ravi Gummadi
>            Assignee: Ravi Gummadi
>         Attachments: d_bw.patch, d_bw.v1.patch, d_bw.v2.patch
>
>
> distcp should support an option for user to specify the bandwidth limit for the distcp job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.