hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvina Caíno Lores <silvi.ca...@gmail.com>
Subject Re: How to configure multiple reduce jobs in hadoop 2.2.0
Date Fri, 17 Jan 2014 09:26:31 GMT
Also, you should be limited by your container configuration at
yarn-site.xml and mapred-site.xml, check
THIS<http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/>to
understand how resource management works.

Basically you can set the number of reducers you want but you are limited
to the number the system can actually hold by the configuration you have
set.

Hope it helps.
Regards,
Silvina


On 16 January 2014 08:54, sudhakara st <sudhakara.st@gmail.com> wrote:

> Hello Ashish,
>
> Using “-D mapreduce.job.reduces=number” with fixed number of reducer will
> spawn that many for a job.
>
>
> On Thu, Jan 16, 2014 at 12:45 PM, Ashish Jain <ashjain2@gmail.com> wrote:
>
>> Dear All,
>>
>> I have a 3 node cluster and have a map reduce job running on it. I have 8
>> data blocks spread across all the 3 nodes. While running map reduce job I
>> could see 8 map tasks running however reduce job is only 1. Is there a way
>> to configure multiple reduce jobs?
>>
>> --Ashish
>>
>
>
>
> --
>
> Regards,
> ...Sudhakara.st
>
>

Mime
View raw message