hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject RE: How to control the number of map tasks for each nodes?
Date Fri, 09 Jul 2010 01:35:31 GMT

Just a nit.

i7s are quad core. While Intel has 2 'threads' per core. (I'm assuming you're using the i7's
(800 series and below) in a single socket mother board.)
Based on experience with tuning other databases, I am a bit skeptical about going beyond the
number of cores. Also your memory is probably going to be more of a driver on the number of
map/reduce tasks you can run simultaneously.

JMHO, YMMV.

-Mike


> Date: Fri, 9 Jul 2010 10:14:14 +0900
> Subject: Re: How to control the number of map tasks for each nodes?
> From: mp2893@gmail.com
> To: common-user@hadoop.apache.org
> 
> Thanks you all for the info.
> 
> I have successfully configured my cluster.
> Thanks again.
> 
> 2010/7/8 edward choi <mp2893@gmail.com>
> 
> > Hi,
> >
> > I have a cluster consisting of 11 slaves and a single master.
> >
> > The thing is that 3 of my slaves have i7 cpu which means that they can have
> > up to 8 simultaneous processes.
> > But other slaves only have dual core cpus.
> >
> > So I was wondering if I can specify the number of map tasks for each of my
> > slaves.
> > For example, I want to give 8 map tasks to the slaves that have i7 cpus and
> > only two map tasks to the others.
> >
> > Is there a way to do this?
> >
 		 	   		  
_________________________________________________________________
The New Busy is not the old busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message