hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kartheek muthyala <kartheek0...@gmail.com>
Subject Re: Increase number of map slots
Date Wed, 06 Jun 2012 04:26:44 GMT
I think mapred.tasktracker.map.tasks.maximum sets the number of map
tasks and not slots.
Keith, so that means per core, if I specify the maximum number of map
tasks it can host, whether all the maps per slot will run parallely or
they execute one after the other?. I understand that maps that are
scheduled on different cores can run parallely.

Thanks,
Kartheek.

On Tue, Jun 5, 2012 at 11:05 AM, madhu phatak <phatak.dev@gmail.com> wrote:
> Hi,
>  You can change the no of slots using the following properties in
> mapred-site.xml.
>
>  mapred.tasktracker.map.tasks.maximum ( for map slots)
>  mapred.tasktracker.reduce.tasks.maximum (for reduce slot)
>
>
> On Tue, Jun 5, 2012 at 3:44 AM, Keith Wiley <kwiley@keithwiley.com> wrote:
>>
>> The number of slots corresponds to the number of cores (e.g., a 10-node
>> cluster consisting of quad-core cpus has 40 cores and therefore 40 slots).
>>  The slots can be divided amongst the maps and reduces in a way which
>> integer-divides into the cores of a single machine.  So if the nodes are
>> quad-cores you can do 3:1, 2:2, or 1:3.  If the nodes are dual-quad-cores
>> (each node has eight cores) you can do 7:1 through 1:7 in any configuration
>> you want..........as per my understanding.  Others may pipe in to clarify
>> some minor point I have missed, I'm unsure.
>>
>> On Jun 4, 2012, at 09:06 , kartheek muthyala wrote:
>>
>> > Hello all,
>> >
>> > Can I increase the number of map slots in a machine using some
>> > configuration?. I can find configurations for increasing the number of
>> > maps and reduce tasks that can run, but didn't find anything related
>> > to slots. Is that default decided based on the memory size?
>> >
>> > Thanks,
>> > Kartheek.
>>
>>
>>
>> ________________________________________________________________________________
>> Keith Wiley     kwiley@keithwiley.com     keithwiley.com
>>  music.keithwiley.com
>>
>> "I do not feel obliged to believe that the same God who has endowed us
>> with
>> sense, reason, and intellect has intended us to forgo their use."
>>                                           --  Galileo Galilei
>>
>> ________________________________________________________________________________
>>
>
>
>
> --
> https://github.com/zinnia-phatak-dev/Nectar
>

Mime
View raw message