hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chuck Lan" <c...@modeln.com>
Subject RE: how to increase number of reduce tasks
Date Thu, 31 Jul 2008 19:40:35 GMT
sorry, mapred.reduce.tasks.  Maybe you need to restart the namenode?

-----Original Message-----
From: Chuck Lan [mailto:clan@modeln.com] 
Sent: Thursday, July 31, 2008 12:39 PM
To: core-user@hadoop.apache.org
Subject: RE: how to increase number of reduce tasks

I just set the property map.reduce.tasks.


-----Original Message-----
From: Alexander Aristov [mailto:alexander.aristov@gmail.com] 
Sent: Thursday, July 31, 2008 12:07 PM
To: core-user@hadoop.apache.org
Subject: how to increase number of reduce tasks


I am running nutch on hadoop 0.17.1. I launch 5 nodes to perform

When I look at the job statistics I see that only 1 reduce task is
for all steps and hence I do a conclusion that hadoop doesn't consume
available resources.

Only one node is extremily busy, other nodes are idle. How can I
hadoop to consume all resources?

I added mapred.map.tasks and mapred.reduce.tasks parameters but they
have no
I also increased the max number for the mapred tasks, job tracker shows

During all stages map tasks  reaches maximum 3, andreduce only 1.

Best Regards
Alexander Aristov

View raw message