hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinsong Hu" <jinsong...@hotmail.com>
Subject is there any way to set the niceness of map/reduce jobs
Date Tue, 02 Nov 2010 23:51:31 GMT
Hi: there:
  I have a cluster that is used for both hadoop mapreduce and hbase. What I 
found is that when I am running map/reduce jobs, the job can be very 
memory/cpu intensive, and cause hbase or data nodes to freeze. in hbase's 
case, the region server may shut it self down.
  In order to avoid this, I made very conservative configuration of the 
maximum number of mappers and reducers. However, I am wonder if hadoop 
allows me to start map/reduce with the command "nice" so that
those jobs get lower priority than datanode/tasktracker/hbase regionserver. 
That way, if there is enough resource, the jobs can fully utilize them. but 
if not, those jobs will yield to other processes.


View raw message