hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-7070) Batch calls to RackResolver from AM
Date Fri, 30 Mar 2018 08:10:00 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-7070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Greg Phillips updated MAPREDUCE-7070:
    Status: Patch Available  (was: Open)

> Batch calls to RackResolver from AM
> -----------------------------------
>                 Key: MAPREDUCE-7070
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7070
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster, performance
>            Reporter: Greg Phillips
>            Priority: Minor
>         Attachments: MAPREDUCE-7070.patch
> TaskAttemptImpl calls RackResolver once for every split location. This results in many
shells being spawned by the Application Master when the topology script is enabled. The worst
case scenario would entail one call to the topology script for each Datanode. The NameNode
is also topology aware, and uses batch calls to the topology script to improve performance.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message