Return-Path: X-Original-To: apmail-ambari-user-archive@www.apache.org Delivered-To: apmail-ambari-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6269C18DE4 for ; Tue, 13 Oct 2015 09:29:37 +0000 (UTC) Received: (qmail 46275 invoked by uid 500); 13 Oct 2015 09:29:37 -0000 Delivered-To: apmail-ambari-user-archive@ambari.apache.org Received: (qmail 46246 invoked by uid 500); 13 Oct 2015 09:29:37 -0000 Mailing-List: contact user-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ambari.apache.org Delivered-To: mailing list user@ambari.apache.org Received: (qmail 46236 invoked by uid 99); 13 Oct 2015 09:29:37 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Oct 2015 09:29:37 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id AC275C084E for ; Tue, 13 Oct 2015 09:29:36 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.998 X-Spam-Level: **** X-Spam-Status: No, score=4.998 tagged_above=-999 required=6.31 tests=[FSL_HELO_BARE_IP_2=1.999, HTML_MESSAGE=3, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id mfQMA6NYUqk4 for ; Tue, 13 Oct 2015 09:29:34 +0000 (UTC) Received: from relayvx11b.securemail.intermedia.net (relayvx11b.securemail.intermedia.net [64.78.52.184]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 9C8CA42B30 for ; Tue, 13 Oct 2015 09:29:34 +0000 (UTC) Received: from securemail.intermedia.net (localhost [127.0.0.1]) by emg-ca-1-1.localdomain (Postfix) with ESMTP id DC01B53E93 for ; Tue, 13 Oct 2015 02:29:33 -0700 (PDT) Subject: Re: Unable to set the namenode options using blueprints MIME-Version: 1.0 x-echoworx-msg-id: cd4c7e2b-3cdc-46b2-a5c0-f8b6092cda66 x-echoworx-emg-received: Tue, 13 Oct 2015 02:29:33.844 -0700 x-echoworx-action: delivered Received: from 10.254.155.14 ([10.254.155.14]) by emg-ca-1-1 (JAMES SMTP Server 2.3.2) with SMTP ID 590 for ; Tue, 13 Oct 2015 02:29:33 -0700 (PDT) Received: from MBX080-W4-CO-2.exch080.serverpod.net (unknown [10.224.117.102]) by emg-ca-1-1.localdomain (Postfix) with ESMTP id A12EF53E93 for ; Tue, 13 Oct 2015 02:29:33 -0700 (PDT) Received: from MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) by MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Tue, 13 Oct 2015 02:29:32 -0700 Received: from MBX080-W4-CO-2.exch080.serverpod.net ([10.224.117.102]) by mbx080-w4-co-2.exch080.serverpod.net ([10.224.117.102]) with mapi id 15.00.1044.021; Tue, 13 Oct 2015 02:29:32 -0700 From: Dmitry Sen To: "user@ambari.apache.org" Thread-Topic: Unable to set the namenode options using blueprints Thread-Index: AQHRBYI3lDFe7RS760q0s0DEAV+wEp5pJwTw Date: Tue, 13 Oct 2015 09:29:32 +0000 Message-ID: <1444728572240.52112@hortonworks.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [109.86.55.247] x-source-routing-agent: Processed Content-Type: multipart/alternative; boundary="_000_144472857224052112hortonworkscom_" --_000_144472857224052112hortonworkscom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable hadoop-env has no property HADOOP_NAMENODE_OPTS?, you should use namenode_o= pt_maxnewsize for specifying XX:MaxHeapSize? "hadoop-env" : { "properties" : { "namenode_opt_maxnewsize" : "16384m" } } You may also want to check all available options in /var/lib/ambari-agent/c= ache/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml? ________________________________ From: Stephen Boesch Sent: Tuesday, October 13, 2015 9:41 AM To: user@ambari.apache.org Subject: Unable to set the namenode options using blueprints Given a blueprint that includes the following: "hadoop-env" : { "properties" : { "HADOOP_NAMENODE_OPTS" : " -XX:InitialHeapSize=3D16384m -XX:Max= HeapSize=3D16384m -Xmx16384m -XX:MaxPermSize=3D512m" } } The following occurs when creating the cluster: Error occurred during initialization of VM Too small initial heap The logs say: CommandLine flags: -XX:ErrorFile=3D/var/log/hadoop/hdfs/hs_err_pid%p.log -X= X:InitialHeapSize=3D1024 -XX:MaxHeapSize=3D1024 -XX:MaxNewSize=3D200 -XX:Ma= xTenuringThreshold=3D6 -XX:NewSize=3D200 -XX:OldPLABSize=3D16 -XX:OnOutOfMe= moryError=3D"/usr/hdp/current/hadoop-hdfs-namenode/bin/kill-name-node" -XX:= OnOutOfMemoryError=3D"/usr/hdp/current/hadoop-hdfs-namenode/bin/kill-name-n= ode" -XX:OnOutOfMemoryError=3D"/usr/hdp/current/hadoop-hdfs-namenode/bin/ki= ll-name-node" -XX:ParallelGCThreads=3D8 -XX:+PrintGC -XX:+PrintGCDateStamps= -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers= -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC Notice that nowhere are the options provided included in the actual jvm lau= nched values. it is no wonder the low on resources given the only 1GB MaxHeapSize. totall= y inadequate for namenode. btw this is HA - and both of the namenodes have same behavior. --_000_144472857224052112hortonworkscom_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

hadoop-env has no property HADOOP_NAMENODE_OPTS​, you sho= uld use namenode_opt_maxnewsize for specifying XX:MaxHeapSize​  

      "hadoop-env" : {
        "properties" : {
           "namenode_opt_maxnewsize" :  "1638= 4m"
        }
      }


You may also want to check all available options in /var/lib/ambari= -agent/cache/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml= 203;


From: Stephen Boesch <ja= vadba@gmail.com>
Sent: Tuesday, October 13, 2015 9:41 AM
To: user@ambari.apache.org
Subject: Unable to set the namenode options using blueprints
 
Given a blueprint that includes the following:

      "hadoop-env" : {
        "properties" : {
           "HADOOP_NAMENODE_OPTS&qu= ot; :  " -XX:InitialHeapSize=3D16384m -XX:MaxHeapSize=3D16384m -X= mx16384m -XX:MaxPermSize=3D512m"
        }
      }
 
The following occurs when creating the cluster:

Error occurred during initialization of VM
Too small initial heap

The logs say:

CommandLine flags: -XX:ErrorFile=3D/var/log/hadoop/hdfs/hs_err_pid%p.l= og -XX:InitialHeapSize=3D1024 -XX:MaxHeapSize=3D1024 -XX:MaxNewSize=3D200 -XX:MaxTenuringThreshold= =3D6 -XX:NewSize=3D200 -XX:OldPLABSize=3D16 -XX:OnOutOfMemoryError=3D"= /usr/hdp/current/hadoop-hdfs-namenode/bin/kill-name-node" -XX:OnOutOfM= emoryError=3D"/usr/hdp/current/hadoop-hdfs-namenode/bin/kill-name-node= " -XX:OnOutOfMemoryError=3D"/usr/hdp/current/hadoop-hdfs-namenode/bin/k= ill-name-node" -XX:ParallelGCThreads=3D8 -XX:+PrintGC -XX:+Pri= ntGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+= UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkS= weepGC -XX:+UseParNewGC

Notice that nowhere are the options provided included in the actual jv= m launched values.


it is no wonder the low on resources given the only 1GB MaxHeapSize. t= otally inadequate for namenode.

btw this is HA - and both of the namenodes have same behavior.



--_000_144472857224052112hortonworkscom_--