hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eduard Skaley <e.v.ska...@gmail.com>
Subject Controlling on which node a reducer will be executed
Date Mon, 27 Aug 2012 17:11:21 GMT
Hi,

i have a question concerning the execution of reducers.
To use effectively the data locality of blocks in my use case i want to
control on which node a reducer will be executed.

In my scenario i have a chain of map-reduce jobs where each job will be
executed by exactly N reducers.
I want to achieve that for each job reducer 1 is executed on node 1,
reducer 2 on node 2 and so on.

Is it possible with a customization of the BlockPlacementPolicy in MRv2 ?
Or is there some other way, maybe some indirect ? Or is it impossible ?

Thanks for your help.
Eduard

Mime
View raw message