hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dieter De Witte <drdwi...@gmail.com>
Subject Re: mapred.tasktacker.reduce.tasks.maximum issue
Date Thu, 14 Nov 2013 15:02:00 GMT
you could make the values final: add <final> true </final> to make sure
they cannot be overridden..


2013/11/14 Vincent Y. Shen <vincent.y.shen@gmail.com>

> Hi Dieter,
>
> Thanks a lot for prompt reply! very much appreciated.
>
> Indeed, I did change in the mapred-site.xml in ALL the nodes and restarted
> the cluster 4 times but no luck...
>
>
> Any other hint?
>
>
>
>
>
>
> On Thu, Nov 14, 2013 at 2:56 PM, Dieter De Witte <drdwitte@gmail.com>wrote:
>
>> Dear Vincent,
>>
>> You have to make sure that mapred-site is modified on every tasktracker
>> node! Not just the master! Mapred-site can be customized if you have a
>> heterogeneous cluster. I usually start the dfs, then copy all the
>> mapredsites to the slave nodes, then run start-mapred.sh and then start a
>> job.
>>
>> Regards, Dieter
>>
>>
>> 2013/11/14 Vincent Y. Shen <vincent.y.shen@gmail.com>
>>
>> Hi Guys,
>>>
>>> I am increasing mapred.tasktacker.reduce.tasks.maximum in the hadoop
>>> cluster. But regardless whatever I tried, hadoop refuses to use my value
>>> but the default (2)
>>>
>>>
>>>
>>>
>>> What I did do
>>> ===========
>>>
>>> stop mapred
>>> stop dfs
>>>
>>> update the value of mapred.tasktacker.reduce.tasks.maximum
>>>
>>>
>>> start dfs
>>> start mapred
>>>
>>>
>>> But the system still uses default values
>>>
>>>
>>>
>>>
>>> Environment
>>> =========
>>>
>>> Hadoop version: 1.1.2
>>> Java version: 7
>>>
>>>
>>> Mapred-site.xml
>>> ============
>>>
>>> <property>
>>>   <name>mapred.system.dir</name>
>>>   <value>/hadoop/mapred/system</value>
>>> </property>
>>>
>>>
>>>
>>> <property>
>>>   <name>mapred.reduce.tasks</name>
>>>   <value>50</value>
>>> </property>
>>>
>>> <property>
>>>   <name>mapred.tasktacker.reduce.tasks.maximum</name>
>>>   <value>15</value>
>>> </property>
>>>
>>> <property>
>>>   <name>mapred.tasktacker.map.tasks.maximum</name>
>>>   <value>15</value>
>>> </property>
>>>
>>>
>>> <property>
>>>   <name>mapred.child.java.opts</name>
>>>   <value>-Xmx500m</value>
>>> </property>
>>>
>>>
>>>
>>> <property>
>>>   <name>mapred.job.tracker</name>
>>>   <value>machine1.mydomain.com:54311</value>
>>>
>>> </property>
>>>
>>>
>>>
>>> Thanks guys for any pointers!!!!
>>>
>>>
>>> Best regards,
>>>
>>>
>>> Vincent.
>>>
>>>
>>>
>>
>

Mime
View raw message