hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sudhakara st <sudhakara...@gmail.com>
Subject Re: How to configure multiple reduce jobs in hadoop 2.2.0
Date Fri, 17 Jan 2014 09:46:28 GMT
Also check this
http://blog.cloudera.com/blog/2013/11/migrating-to-mapreduce-2-on-yarn-for-operators/


On Fri, Jan 17, 2014 at 2:56 PM, Silvina Caíno Lores
<silvi.caino@gmail.com>wrote:

> Also, you should be limited by your container configuration at
> yarn-site.xml and mapred-site.xml, check THIS<http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/>to
understand how resource management works.
>
> Basically you can set the number of reducers you want but you are limited
> to the number the system can actually hold by the configuration you have
> set.
>
> Hope it helps.
> Regards,
> Silvina
>
>
> On 16 January 2014 08:54, sudhakara st <sudhakara.st@gmail.com> wrote:
>
>> Hello Ashish,
>>
>> Using “-D mapreduce.job.reduces=number” with fixed number of reducerwill spawn
that many for a job.
>>
>>
>> On Thu, Jan 16, 2014 at 12:45 PM, Ashish Jain <ashjain2@gmail.com> wrote:
>>
>>> Dear All,
>>>
>>> I have a 3 node cluster and have a map reduce job running on it. I have
>>> 8 data blocks spread across all the 3 nodes. While running map reduce job I
>>> could see 8 map tasks running however reduce job is only 1. Is there a way
>>> to configure multiple reduce jobs?
>>>
>>> --Ashish
>>>
>>
>>
>>
>> --
>>
>> Regards,
>> ...Sudhakara.st
>>
>>
>
>


-- 

Regards,
...Sudhakara.st

Mime
View raw message