hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alvin Chyan <alvin.ch...@turn.com>
Subject mapper re-run behavior when node dies
Date Tue, 12 Apr 2016 18:32:30 GMT
Hi all,
If a mapper finished and the output has already been copied to a reducer,
and then the node dies, do the mappers that outputted data to the local
node have to be re-run?

The use case is running mappers on flaky/preemptible nodes. In a cloud
environment, preemptible nodes can be much cheaper. If the mappers always
have to re-run when the node dies, then we may never make progress as
machines can take turns dying and take the output of all map tasks that ran
on that node with it.

I have done some testing, and it seems that if the reducer has finished the
shuffle and has started reducing, then it doesn't matter if the mapper node
died. However, while the reducer was still in copy phase, I killed another
machine and then a bunch of mappers had to re-run. I'm not sure how to
confirm that the data had been successfully copied to reducer though.

Answers or pointers to relevant portions of the code would be greatly


*Alvin Chyan*Sr. Software Engineer, Data
901 Marshall St, Suite 200, Redwood City, CA 94063

turn.com <http://www.turn.com/>   |   @TurnPlatform

This message is Turn Confidential, except for information included that is
already available to the public. If this message was sent to you
accidentally, please delete it.

View raw message