hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Costa <psdc1...@gmail.com>
Subject Re: How set number of map and reduce can run simultaneously
Date Tue, 24 May 2011 11:01:00 GMT
I found the solution. The problem was that I've misspelled the
parameter "mapred.tasktracker.map.tasks.maximum".

On Tue, May 24, 2011 at 11:06 AM, Pedro Costa <psdc1978@gmail.com> wrote:
> I think it's important to say that it exists 2 cpus per node and 12
> core(s) per cpu.
>
> On Tue, May 24, 2011 at 11:02 AM, Pedro Costa <psdc1978@gmail.com> wrote:
>> And all the nodes have the same configuration. A job has 5000 map tasks.
>>
>> On Tue, May 24, 2011 at 10:57 AM, Pedro Costa <psdc1978@gmail.com> wrote:
>>> The values are:
>>> #map tasks: 8
>>> #reduce tasks: 10
>>> Map task capacity:10
>>> Reduce task capacity:10
>>>
>>>
>>> On Tue, May 24, 2011 at 8:01 AM, Harsh J <harsh@cloudera.com> wrote:
>>>> How many tasks are present in your job? Do all tasktrackers carry this
>>>> configuration? What is the total reported slot capacity on the JT UI?
>>>>
>>>> On Mon, May 23, 2011 at 10:28 PM, Pedro Costa <psdc1978@gmail.com>
wrote:
>>>>> I think I've to rephrase the question.
>>>>>
>>>>> I set the "mapred.tasktracker.map.tasks.maximum" to 8, hoping that it
>>>>> will run 8*10 map tasks in the whole cluster. But, it only run 8 tasks
>>>>> simultaneously. Why this is happens?
>>>>>
>>>>> On Mon, May 23, 2011 at 5:45 PM, Pedro Costa <psdc1978@gmail.com>
wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I'm running hadoop map-reduce in a cluster with 10 machines. I would
>>>>>> like to set in the configuration that each tasktracker can run 8
map
>>>>>> tasks simultaneously and 4 reduce tasks simultaneously.  Which
>>>>>> parameters should I configure?
>>>>>>
>>>>>> Thanks,
>>>>>> PSC
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Harsh J
>>>>
>>>
>>>
>>>
>>
>
>
>
> --
> ---------------------------
> Pedro Sá da Costa
>
> @: pcosta@lasige.di.fc.ul.pt
> @: psdc1978@gmail.com
>



-- 
---------------------------
Pedro Sá da Costa

@: pcosta@lasige.di.fc.ul.pt
@: psdc1978@gmail.com

Mime
View raw message