hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghava Mutharaju <m.vijayaragh...@gmail.com>
Subject avoiding data redistribution in iterative mapreduce
Date Wed, 03 Feb 2010 06:04:57 GMT
Hi all,

      I to run a map reduce task repeatedly in order to achieve the desired
result. Is it possible that at the beginning of each iteration, the data set
is not distributed (divided into chunks and distributed) again and again
i.e. once the distribution occurs for the first time, map nodes should work
on the same chunk in every iteration. Can this be done? I only have a brief
experience with MapReduce and I think that the input data set is
redistributed every time.

Thank you.

Regards,
Raghava.

Mime
View raw message