hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Lilley <john.lil...@redpoint.net>
Subject MapReduce task-worker assignment
Date Sat, 05 Oct 2013 22:12:01 GMT
Is there a description of how MapReduce under Hadoop 2.0 assigns mapper tasks to preferred
nodes?  I think that someone on the list mentioned previously that it attempted to assign
"one HDFS block per mapper task", but given that there can be multiple block instances per
data split, how does MapReduce try to obtain an even task assignment while optimizing data
locality?
Thanks,
John Lilley
Chief Architect, RedPoint Global Inc.
1515 Walnut Street | Suite 200 | Boulder, CO 80302
T: +1 303 541 1516  | M: +1 720 938 5761 | F: +1 781-705-2077
Skype: jlilley.redpoint | john.lilley@redpoint.net<mailto:john.lilley@redpoint.net>
| www.redpoint.net<http://www.redpoint.net/>


Mime
View raw message