hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vincent Y. Shen" <vincent.y.s...@gmail.com>
Subject Re: mapred.tasktacker.reduce.tasks.maximum issue
Date Thu, 14 Nov 2013 14:58:58 GMT
Hi Dieter,

Thanks a lot for prompt reply! very much appreciated.

Indeed, I did change in the mapred-site.xml in ALL the nodes and restarted
the cluster 4 times but no luck...


Any other hint?






On Thu, Nov 14, 2013 at 2:56 PM, Dieter De Witte <drdwitte@gmail.com> wrote:

> Dear Vincent,
>
> You have to make sure that mapred-site is modified on every tasktracker
> node! Not just the master! Mapred-site can be customized if you have a
> heterogeneous cluster. I usually start the dfs, then copy all the
> mapredsites to the slave nodes, then run start-mapred.sh and then start a
> job.
>
> Regards, Dieter
>
>
> 2013/11/14 Vincent Y. Shen <vincent.y.shen@gmail.com>
>
> Hi Guys,
>>
>> I am increasing mapred.tasktacker.reduce.tasks.maximum in the hadoop
>> cluster. But regardless whatever I tried, hadoop refuses to use my value
>> but the default (2)
>>
>>
>>
>>
>> What I did do
>> ===========
>>
>> stop mapred
>> stop dfs
>>
>> update the value of mapred.tasktacker.reduce.tasks.maximum
>>
>>
>> start dfs
>> start mapred
>>
>>
>> But the system still uses default values
>>
>>
>>
>>
>> Environment
>> =========
>>
>> Hadoop version: 1.1.2
>> Java version: 7
>>
>>
>> Mapred-site.xml
>> ============
>>
>> <property>
>>   <name>mapred.system.dir</name>
>>   <value>/hadoop/mapred/system</value>
>> </property>
>>
>>
>>
>> <property>
>>   <name>mapred.reduce.tasks</name>
>>   <value>50</value>
>> </property>
>>
>> <property>
>>   <name>mapred.tasktacker.reduce.tasks.maximum</name>
>>   <value>15</value>
>> </property>
>>
>> <property>
>>   <name>mapred.tasktacker.map.tasks.maximum</name>
>>   <value>15</value>
>> </property>
>>
>>
>> <property>
>>   <name>mapred.child.java.opts</name>
>>   <value>-Xmx500m</value>
>> </property>
>>
>>
>>
>> <property>
>>   <name>mapred.job.tracker</name>
>>   <value>machine1.mydomain.com:54311</value>
>>
>> </property>
>>
>>
>>
>> Thanks guys for any pointers!!!!
>>
>>
>> Best regards,
>>
>>
>> Vincent.
>>
>>
>>
>

Mime
View raw message