Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1855546410.1210232455741.JavaMail.jira@brutus>
Date: Thu, 8 May 2008 00:40:55 -0700 (PDT)
From: "Amar Kamat (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3296) Some levels are skipped while
 creating the task cache in JobInProgress
In-Reply-To: <1637218255.1208847081408.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595138#action_12595138 ] 

Amar Kamat commented on HADOOP-3296:
------------------------------------

bq. Minimally, calling the total number of maps scheduled "Local map tasks" is confusing
Note that the counter will be incremented only if there is a locality at any level. Its different from calling all the maps as LOCAL. I agree that we should change the counter name. The intention is to show how many tasks got picked up from the cache. So it includes node-local, rack-local, switch-local etc. I think _Aggregate Locality_ sounds better, comments?
bq. For which users would this be valuable information?
In general when the users have cache topology more than 2 levels.
bq. If there are more than two levels in the task cache and the distinction is significant, how is an aggregate counter resolving the ambiguity?
Consider a case where there is no locality at the node level and also at the rack level. As per the trunk there is no way to infer whether the scheduling went correctly or not. With this aggregate counter one can check if the maps were from the cache or not. So the counter is just a count of how may maps got picked up from the task cache.

> Some levels are skipped while creating the task cache in JobInProgress
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-3296
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3296
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.17.0
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: HADOOP-3296-v2.patch, HADOOP-3296.patch
>
>
> Consider the following piece of code
> {code:title=JobInProgress.createCache()|borderStyle=solid}
> Node node = jobtracker.resolveAndAddToTopology(host);
> for (int j = 0; j < maxLevel; j++) {
>           node = JobTracker.getParentNode(node, j);
>           .....
> {code}
> With {{maxLevel > 2}} the caches will be created in the following order
> ||j||node-level||
> |0|0|
> |1|1|
> |2|3|
> |3|6|
> which is not as desired.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.