Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C69051764E for ; Thu, 9 Oct 2014 17:48:59 +0000 (UTC) Received: (qmail 29804 invoked by uid 500); 9 Oct 2014 17:48:52 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 29701 invoked by uid 500); 9 Oct 2014 17:48:52 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 29690 invoked by uid 99); 9 Oct 2014 17:48:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Oct 2014 17:48:51 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of sfhadoop@gmail.com designates 209.85.215.53 as permitted sender) Received: from [209.85.215.53] (HELO mail-la0-f53.google.com) (209.85.215.53) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Oct 2014 17:48:23 +0000 Received: by mail-la0-f53.google.com with SMTP id gq15so1687577lab.40 for ; Thu, 09 Oct 2014 10:48:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=wHDzPdBr1h2OEo2gDQv6W8E8gdwBPxQfAktbkHGbyDI=; b=elrKNY1vO+Qg0UjVBunDKOYmN/g3RAMdLBjs35syl0n/xW6xgjqJEcXaWGBveRNq8h HdaZLJQjZleQwN1AMns52btQG67h4Q4M9GhXG6RRZVFkPQxWZeU5XIDYinzKF9C9vNqp //Nx7zIuztOfXtZ0pEdfxz/As+8qwS2e8E6ZD9cHSg+IPdCzptowVQ3fU/AEfyj0jmJ4 HwTib0nyYk83FlUk2uWkVynF/xDQ80ecU1YDdFqV3gPG9ckE11mvw2GEx12zzKUUXWGu QVbpMM56SaN+ctYy1QTeyFVsFbUMSHq6j1hcey8WgZv6RqPUbB4CARcXdnxJEX3bDLc7 vsgQ== MIME-Version: 1.0 X-Received: by 10.112.150.230 with SMTP id ul6mr11381093lbb.103.1412876903014; Thu, 09 Oct 2014 10:48:23 -0700 (PDT) Received: by 10.25.31.84 with HTTP; Thu, 9 Oct 2014 10:48:22 -0700 (PDT) In-Reply-To: References: Date: Thu, 9 Oct 2014 10:48:22 -0700 Message-ID: Subject: Re: Hadoop configuration for cluster machines with different memory capacity / # of cores etc. From: SF Hadoop To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=047d7b3435eaea67a90505010982 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b3435eaea67a90505010982 Content-Type: text/plain; charset=UTF-8 Yes. You are correct. Just keep in mind, for every spec X machine you have to have version X of hadoop configs (that only reside on spec X machines). Version Y configs reside on only version Y machines, and so on. But yes, it is possible. On Thu, Oct 9, 2014 at 9:40 AM, Manoj Samel wrote: > So, in that case, the resource manager will allocate containers of > different capacity based on node capacity ? > > Thanks, > > On Wed, Oct 8, 2014 at 9:42 PM, Nitin Pawar > wrote: > >> you can have different values on different nodes >> >> On Thu, Oct 9, 2014 at 4:15 AM, Manoj Samel >> wrote: >> >>> In a hadoop cluster where different machines have different memory >>> capacity and / or different # of cores etc., it is required that >>> memory/core related parameters be set to SAME for all nodes ? Or it is >>> possible to set different values for different nodes ? >>> >>> E.g. can yarn.nodemanager.resource.memory-mb >>> and yarn.nodemanager.resource.cpu-vcores have different values for >>> different nodes ? >>> >>> Thanks, >>> >>> >>> >> >> >> -- >> Nitin Pawar >> > > --047d7b3435eaea67a90505010982 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Yes.=C2=A0 You are correct.=C2=A0 Just keep in mind, for e= very spec X machine you have to have version X of hadoop configs (that only= reside on spec X machines).=C2=A0 Version Y configs reside on only version= Y machines, and so on.

But yes, it is possible.

On Thu, Oct 9= , 2014 at 9:40 AM, Manoj Samel <manojsameltech@gmail.com> wrote:
So, in that = case, the resource manager will allocate containers of different capacity b= ased on node capacity ?

Thanks,

On Wed, Oct 8, 2014 at 9:42 PM= , Nitin Pawar <nitinpawar432@gmail.com> wrote:
you can have different values o= n different nodes=C2=A0

On Thu, Oct 9, 2014 at 4:15 AM, Manoj Samel <= manojsameltech@gmail.com> wrote:
In a hadoop cluster where different machines have di= fferent memory capacity and / or different # of cores etc., it is required = that memory/core related parameters be set to SAME for all nodes ? Or it is= possible to set different values for different nodes ?

= E.g. can yarn.nodemanager.resource.memory-mb and=C2=A0yarn.nodemanager.reso= urce.cpu-vcores have different values for different nodes ?
<= br>
Thanks,





--
Nitin Pawar<= br>


--047d7b3435eaea67a90505010982--