hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3296) Some levels are skipped while creating the task cache in JobInProgress
Date Wed, 07 May 2008 04:53:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12594776#action_12594776
] 

Amar Kamat commented on HADOOP-3296:
------------------------------------

bq. but I don't understand the purpose of the new counter (it could also use an entry in JobInProgress_Counter.properties,
btw) ....
The reason is that if there are more than 2 levels in the task cache (e.g local/rack/gateway/...)
then there is no way to know if the maps were scheduled locally (at a level above rack) or
not. Thanks for the review.

> Some levels are skipped while creating the task cache in JobInProgress
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-3296
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3296
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.17.0
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: HADOOP-3296.patch
>
>
> Consider the following piece of code
> {code:title=JobInProgress.createCache()|borderStyle=solid}
> Node node = jobtracker.resolveAndAddToTopology(host);
> for (int j = 0; j < maxLevel; j++) {
>           node = JobTracker.getParentNode(node, j);
>           .....
> {code}
> With {{maxLevel > 2}} the caches will be created in the following order
> ||j||node-level||
> |0|0|
> |1|1|
> |2|3|
> |3|6|
> which is not as desired.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message