hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-7070) Batch calls to RackResolver from AM
Date Thu, 29 Mar 2018 15:16:00 GMT
Greg Phillips created MAPREDUCE-7070:
----------------------------------------

             Summary: Batch calls to RackResolver from AM
                 Key: MAPREDUCE-7070
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7070
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: applicationmaster, performance
            Reporter: Greg Phillips


TaskAttemptImpl calls RackResolver once for every split location. This results in many shells
being spawned by the Application Master when the topology script is enabled. The worst case
scenario would entail one call to the topology script for each Datanode. The NameNode is also
topology aware, and uses batch calls to the topology script to improve performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message