hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Costa <psdc1...@gmail.com>
Subject Re: How set number of map and reduce can run simultaneously
Date Tue, 24 May 2011 10:06:40 GMT
I think it's important to say that it exists 2 cpus per node and 12
core(s) per cpu.

On Tue, May 24, 2011 at 11:02 AM, Pedro Costa <psdc1978@gmail.com> wrote:
> And all the nodes have the same configuration. A job has 5000 map tasks.
>
> On Tue, May 24, 2011 at 10:57 AM, Pedro Costa <psdc1978@gmail.com> wrote:
>> The values are:
>> #map tasks: 8
>> #reduce tasks: 10
>> Map task capacity:10
>> Reduce task capacity:10
>>
>>
>> On Tue, May 24, 2011 at 8:01 AM, Harsh J <harsh@cloudera.com> wrote:
>>> How many tasks are present in your job? Do all tasktrackers carry this
>>> configuration? What is the total reported slot capacity on the JT UI?
>>>
>>> On Mon, May 23, 2011 at 10:28 PM, Pedro Costa <psdc1978@gmail.com> wrote:
>>>> I think I've to rephrase the question.
>>>>
>>>> I set the "mapred.tasktracker.map.tasks.maximum" to 8, hoping that it
>>>> will run 8*10 map tasks in the whole cluster. But, it only run 8 tasks
>>>> simultaneously. Why this is happens?
>>>>
>>>> On Mon, May 23, 2011 at 5:45 PM, Pedro Costa <psdc1978@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I'm running hadoop map-reduce in a cluster with 10 machines. I would
>>>>> like to set in the configuration that each tasktracker can run 8 map
>>>>> tasks simultaneously and 4 reduce tasks simultaneously.  Which
>>>>> parameters should I configure?
>>>>>
>>>>> Thanks,
>>>>> PSC
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
>>
>



-- 
---------------------------
Pedro Sá da Costa

@: pcosta@lasige.di.fc.ul.pt
@: psdc1978@gmail.com

Mime
View raw message