hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: reg max map task config
Date Mon, 28 Jan 2013 16:23:52 GMT
There's probably different variables (esp. since you run HBase,
calculated resources) involved for this, but you can generally find
some reading material on this in books and
http://wiki.apache.org/hadoop/HowManyMapsAndReduces which should help
you come to a good value. Pro-tip I usually get from Todd:
Consistently monitor and tune, for it is harmless to restart a slave
to set in new values.

On Mon, Jan 28, 2013 at 6:04 PM, Jean-Marc Spaggiari
<jean-marc@spaggiari.org> wrote:
> Hi Harsh,
>
> Is there a recommanded way to configure this value? I think I read
> something like 0.7 x cores but I'm not 100% sure...
>
> JM
>
> 2013/1/28, Manoj Babu <manoj444@gmail.com>:
>> Thank you Harsh.
>>
>> Cheers!
>> Manoj.
>>
>>
>> On Mon, Jan 28, 2013 at 10:50 AM, Harsh J <harsh@cloudera.com> wrote:
>>
>>> Hi,
>>>
>>> As noted on
>>> http://hadoop.apache.org/docs/stable/cluster_setup.html#Configuring+the+Hadoop+Daemons
>>> ,
>>> and by its own name, the config 'mapred.tasktracker.map.tasks.maximum'
>>> applies to a TaskTracker daemon and therefore needs to be applied to
>>> its local mapred-site.xml and has to be restarted to take new values
>>> into effect.
>>>
>>> On Mon, Jan 28, 2013 at 10:36 AM, Manoj Babu <manoj444@gmail.com> wrote:
>>> > Hi All,
>>> >
>>> > I am trying to override the value of
>>> mapred.tasktracker.map.tasks.maximum in
>>> > mapred-site.xml through my job configuration object will it reflect in
>>> the
>>> > corresponding task trackers?
>>> > or do i have to change it in mapred-site.xml following a restart for
>>> > the
>>> > changes to apply?
>>> >
>>> > Thanks in advance.
>>> >
>>> > Cheers!
>>> > Manoj.
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>



-- 
Harsh J

Mime
View raw message