hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Hadoop - how exactly is a slot defined
Date Thu, 25 Nov 2010 12:59:50 GMT
On 25/11/10 12:42, Grandl Robert wrote:
> Thanks to you all for the explanations.
> So, as far as I understand, if I configure 4 map slots per node(let's say - 512 MB RAM
per slot as my node has 2 GB in total) the hadoop will always try to allocate 4 slots ?  Does
the node report on the hearbteat that it has 4 free slots ?

> But then, my question comes: what if another workload contend with hadoop workload at
a moment, that means few resources available now for hadoop. Did hadoop still report he has
4 slots free and implicitly try to allocate tasks for these 4 slots ?

Do you mean other system workload? Are your machines accepting work from 
other places?

The JobTracker will push out work to the nodes, and remember which 
machines it has given work to -so won't overcommit them. If you are 
doing other work on the same machine, it won't know about that, and 
still push out jobs that will now take longer.


View raw message