Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 15C7AD6F3 for ; Tue, 13 Nov 2012 15:50:54 +0000 (UTC) Received: (qmail 34112 invoked by uid 500); 13 Nov 2012 15:50:49 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 33950 invoked by uid 500); 13 Nov 2012 15:50:48 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 33924 invoked by uid 99); 13 Nov 2012 15:50:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Nov 2012 15:50:48 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.212.48] (HELO mail-vb0-f48.google.com) (209.85.212.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Nov 2012 15:50:41 +0000 Received: by mail-vb0-f48.google.com with SMTP id e21so3591895vbm.35 for ; Tue, 13 Nov 2012 07:50:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=ztKbJfrwf3VEwjHqvWUcj3J/FrAe9pxEDjhj26vgZEs=; b=Jm2CwU8soDEe2+YoRcbpCwpCC3T5g6SMRAAWA9hSeYkwuavPH9is51oJCzHSo7MSun qkAA7CAMb+YR1DjPielCZ0lY/JEu4n5IIOefxHz6zigY8cz/TGy86nTmdSJ+He9tNWcs t1OZeWqAgcscAGGN7W6h+AMnVzX00nu91C2ImIY4FvKI180kcT7wYTf/9OOWKA/LOX/a MChfrW8kTL05jvR3KSwl71EyNHWzG0KSCkeOO06s6LaZqbEi+pkAyVU5o38xs39Vjg3P Nt0E5fLxv8jyV/ulC9dWMDArhoZl51KoIYKH2xkjKUdo9bU/caqvJaFr9yzDVt0kJ2G0 /jjw== MIME-Version: 1.0 Received: by 10.52.173.74 with SMTP id bi10mr28360261vdc.81.1352821820389; Tue, 13 Nov 2012 07:50:20 -0800 (PST) Received: by 10.220.115.80 with HTTP; Tue, 13 Nov 2012 07:50:19 -0800 (PST) X-Originating-IP: [52.129.8.52] In-Reply-To: <17B4FD88-498D-4A9B-A56E-DEFDF033C49C@gmail.com> References: <17B4FD88-498D-4A9B-A56E-DEFDF033C49C@gmail.com> Date: Tue, 13 Nov 2012 10:50:19 -0500 Message-ID: Subject: Re: mapred.tasktracker.map.tasks.maximum From: Mark Kerzner To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=bcaec51b1bb90c782304ce626075 X-Gm-Message-State: ALoCoQkcvKOfQz8uGvoPvD7pjRRLR+8t+55e+00K4+HCDblIq1QULpvhX6FQUIagPLwgtaKQXFVf X-Virus-Checked: Checked by ClamAV on apache.org --bcaec51b1bb90c782304ce626075 Content-Type: text/plain; charset=ISO-8859-1 I am using a Hadoop manager, and it stores the values in a database, outside the configuration files. On Tue, Nov 13, 2012 at 10:27 AM, Jay Vyas wrote: > Hmmm What do you mean wrong configuration file.? How could that ever > happen? > > Jay Vyas > > On Nov 13, 2012, at 10:25 AM, Mark Kerzner > wrote: > > Exactly! I found the right one, and it is 80. > > Thank you, > Mark > > On Tue, Nov 13, 2012 at 10:23 AM, Serge Blazhiyevskyy < > Serge.Blazhiyevskyy@nice.com> wrote: > >> Look on the job tracker web UI is parameter has taken effect (port 50030) >> >> It might be in the wrong config file or something >> >> >> Serge >> >> From: Mark Kerzner > mark.kerzner@shmsoft.com>> >> Reply-To: "user@hadoop.apache.org" < >> user@hadoop.apache.org> >> Date: Tuesday, November 13, 2012 5:19 PM >> To: "user@hadoop.apache.org" < >> user@hadoop.apache.org> >> Subject: Re: mapred.tasktracker.map.tasks.maximum >> >> 1.0.1 >> >> On Tue, Nov 13, 2012 at 10:18 AM, Serge Blazhiyevskyy < >> Serge.Blazhiyevskyy@nice.com> wrote: >> What hadoop version are we talking about? >> >> From: Mark Kerzner > mark.kerzner@shmsoft.com>> mark.kerzner@shmsoft.com>>> >> Reply-To: "user@hadoop.apache.org> user@hadoop.apache.org>" < >> user@hadoop.apache.org> user@hadoop.apache.org>> >> Date: Tuesday, November 13, 2012 5:16 PM >> To: Hadoop User > >>> >> Subject: mapred.tasktracker.map.tasks.maximum >> >> Hi, >> >> I have a cluster with 4 nodes and 32 many cores on each. My default value >> for the maximum number of mappers per slot is 1: >> >> >> mapred.tasktracker.map.tasks.maximum >> >> 1 >> true >> >> (which I think is wrong). >> >> Howeve, when sizable jobs run, I see 65 mappers working, so it seems that >> it does create more than 1 mapper per node. >> >> Questions: what maximum number of mappers would be appropriate in this >> situation? Is that the right way to set them? >> >> Thank you. Sincerely, >> Mark >> >> > --bcaec51b1bb90c782304ce626075 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I am using a Hadoop manager, and it stores the values in a database, outsid= e the configuration files.

On Tue, Nov 13, 2012 at 10:27 AM, Jay Vyas <= jayunit100@gmail.com> wrote:
Hmmm What do you mean wrong configuration file.? =A0 How could that ev= er happen?

Jay Vyas= =A0=20

On Nov 13, 2012, at 10:25 AM, Mark Kerzner <mark.kerzner@shmsoft.com> wro= te:

Exactly! I found the right one, and it is 80.
=A0
Thank you,
Mark

On Tue, Nov 13, 2012 at 10:23 AM, Serge Blazhiye= vskyy <Serge.Blazhiyevskyy@nice.com> wrote:
Look on the job tracker web UI is par= ameter has taken effect (port 50030)

It might be in the wrong config= file or something


Serge
Date: Tuesday, November 13, 2012 5:19 PM
Subject: Re: mapred.tasktracker.map.tasks.maximum

1.0.1

On Tue, Nov 13, 2012 at 10:18 AM, Serge Blazhiyevskyy <Serge.Blazhiyevs= kyy@nice.com<mailto:Serge.Blazhiyevskyy@nice.com>> wrote:
What hadoop version are we talking about?

From: Mark Kerzner &= lt;mark.kerzn= er@shmsoft.com<mailto:mark.kerzner@shmsoft.com><mailto:mark.kerzner@shmsoft.com&= lt;mailto:mar= k.kerzner@shmsoft.com>>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org= <mailto:user= @hadoop.apache.org>>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org&= gt;<mailto:u= ser@hadoop.apache.org<mailto:user@hadoop.apache.org>>>
Date: Tuesday, November 13, 2012 5:16 PM
To: Hadoop User <= user@hadoop.apa= che.org<mailto:user@hadoop.apache.org><mailto:user@hadoop.apache.org<mailto:user@hadoop.apache.o= rg>>>
Subject: mapred.tasktracker.map.tasks.maximum

Hi,

I have= a cluster with 4 nodes and 32 many cores on each. My default value for the= maximum number of mappers per slot is 1:

=A0 <property>
=A0 =A0 <name>mapred.tasktracker.map.tasks.maximum</name>
= =A0 =A0 <!-- see other kb entry about this one. -->
=A0 =A0 <va= lue>1</value>
=A0 =A0 <final>true</final>
=A0 &l= t;/property>
(which I think is wrong).

Howeve, when sizable jobs run, I see 65 ma= ppers working, so it seems that it does create more than 1 mapper per node.=

Questions: what maximum number of mappers would be appropriate in t= his situation? Is that the right way to set them?

Thank you. Sincerely,
Mark


--bcaec51b1bb90c782304ce626075--