hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From himanshu chandola <himanshu_cool...@yahoo.com>
Subject nodes lying idle
Date Sat, 12 Sep 2009 05:01:42 GMT
Hi everyone,
Ive a cluster of 40 nodes. The input file has 2^18 lines and every line is an input to a map
job. Every node is a quad core and hence I've set mapred.tasktracker.map/reduce.tasks.maximum
to a value greater than 4. The first 20 nodes are showing hadoop jobs taking 100% but with
only one process running while since its a quad core I would've liked to see 4 java processes
taking 100% (there are 5 java processes on this system but 4 are idle and only one is taking
100% or 1 cpu). For the last half of the nodes, the cpu usage of hadoop processes is 0. This
is really strange since my map tasks are processing in a very slow way and I wouldve liked
to use all nodes and all the cores. 

What could possibly be wrong ? It would really help if anyone could suggest .

thanks

H

 Morpheus: Do you believe in fate, Neo?
Neo: No.
Morpheus: Why Not?
Neo: Because I don't like the idea that I'm not in control of my life.



      

Mime
View raw message