hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serge Blazhiyevskyy <Serge.Blazhiyevs...@nice.com>
Subject Re: mapred.tasktracker.map.tasks.maximum
Date Tue, 13 Nov 2012 15:27:20 GMT
;)

From: Mark Kerzner <mark.kerzner@shmsoft.com<mailto:mark.kerzner@shmsoft.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Tuesday, November 13, 2012 5:25 PM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: mapred.tasktracker.map.tasks.maximum

Exactly! I found the right one, and it is 80.

Thank you,
Mark

On Tue, Nov 13, 2012 at 10:23 AM, Serge Blazhiyevskyy <Serge.Blazhiyevskyy@nice.com<mailto:Serge.Blazhiyevskyy@nice.com>>
wrote:
Look on the job tracker web UI is parameter has taken effect (port 50030)

It might be in the wrong config file or something


Serge

From: Mark Kerzner <mark.kerzner@shmsoft.com<mailto:mark.kerzner@shmsoft.com><mailto:mark.kerzner@shmsoft.com<mailto:mark.kerzner@shmsoft.com>>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>>"
<user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>>>
Date: Tuesday, November 13, 2012 5:19 PM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>>"
<user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>>>
Subject: Re: mapred.tasktracker.map.tasks.maximum

1.0.1

On Tue, Nov 13, 2012 at 10:18 AM, Serge Blazhiyevskyy <Serge.Blazhiyevskyy@nice.com<mailto:Serge.Blazhiyevskyy@nice.com><mailto:Serge.Blazhiyevskyy@nice.com<mailto:Serge.Blazhiyevskyy@nice.com>>>
wrote:
What hadoop version are we talking about?

From: Mark Kerzner <mark.kerzner@shmsoft.com<mailto:mark.kerzner@shmsoft.com><mailto:mark.kerzner@shmsoft.com<mailto:mark.kerzner@shmsoft.com>><mailto:mark.kerzner@shmsoft.com<mailto:mark.kerzner@shmsoft.com><mailto:mark.kerzner@shmsoft.com<mailto:mark.kerzner@shmsoft.com>>>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>>>"
<user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>>>>
Date: Tuesday, November 13, 2012 5:16 PM
To: Hadoop User <user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.org>>>>
Subject: mapred.tasktracker.map.tasks.maximum

Hi,

I have a cluster with 4 nodes and 32 many cores on each. My default value for the maximum
number of mappers per slot is 1:

  <property>
    <name>mapred.tasktracker.map.tasks.maximum</name>
    <!-- see other kb entry about this one. -->
    <value>1</value>
    <final>true</final>
  </property>
(which I think is wrong).

Howeve, when sizable jobs run, I see 65 mappers working, so it seems that it does create more
than 1 mapper per node.

Questions: what maximum number of mappers would be appropriate in this situation? Is that
the right way to set them?

Thank you. Sincerely,
Mark



Mime
View raw message