hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keren Ouaknine <ker...@gmail.com>
Subject number of running reducers exceed max number of mappers on cluster
Date Thu, 01 Dec 2011 01:06:00 GMT
Pig script allocates 15 nodes to run a pig script *(how to configure
that?)*but what happens on a 10 nodes cluster only?
UI shows that 15 reducers are running, and at the same time max number of
reducers is 10 (configured one reducer per machine).



Thanks,
Keren

Kind% CompleteNum TasksPendingRunningCompleteKilledFailed/Killed
Task Attempts<http://10.239.24.12:50030/jobfailures.jsp?jobid=job_201111301137_0049>
map<http://10.239.24.12:50030/jobtasks.jsp?jobid=job_201111301137_0049&type=map&pagenum=1>
100.00%
1001<http://10.239.24.12:50030/jobtasks.jsp?jobid=job_201111301137_0049&type=map&pagenum=1&state=completed>
00 / 0 reduce<http://10.239.24.12:50030/jobtasks.jsp?jobid=job_201111301137_0049&type=reduce&pagenum=1>
5.92%

9070<http://10.239.24.12:50030/jobtasks.jsp?jobid=job_201111301137_0049&type=reduce&pagenum=1&state=pending>
15<http://10.239.24.12:50030/jobtasks.jsp?jobid=job_201111301137_0049&type=reduce&pagenum=1&state=running>
5<http://10.239.24.12:50030/jobtasks.jsp?jobid=job_201111301137_0049&type=reduce&pagenum=1&state=completed>
00 / 0
Running Map TasksRunning Reduce TasksTotal Submissions NodesOccupied Map
SlotsOccupied Reduce SlotsReserved Map SlotsReserved Reduce SlotsMap Task
CapacityReduce Task CapacityAvg. Tasks/NodeBlacklisted Nodes Graylisted
NodesExcluded Nodes 004910<http://10.239.24.12:50030/machines.jsp?type=active>
000010102.000 <http://10.239.24.12:50030/machines.jsp?type=blacklisted>
0<http://10.239.24.12:50030/machines.jsp?type=graylisted>
0 <http://10.239.24.12:50030/machines.jsp?type=excluded>



-- 
 Keren Ouaknine
Cell: +972 54 2565404
Web: www.kereno.com

Mime
View raw message