Return-Path: X-Original-To: apmail-bigtop-user-archive@www.apache.org Delivered-To: apmail-bigtop-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B4695CA76 for ; Fri, 28 Nov 2014 19:18:07 +0000 (UTC) Received: (qmail 80334 invoked by uid 500); 28 Nov 2014 19:18:07 -0000 Delivered-To: apmail-bigtop-user-archive@bigtop.apache.org Received: (qmail 80261 invoked by uid 500); 28 Nov 2014 19:18:07 -0000 Mailing-List: contact user-help@bigtop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@bigtop.apache.org Delivered-To: mailing list user@bigtop.apache.org Received: (qmail 80251 invoked by uid 99); 28 Nov 2014 19:18:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Nov 2014 19:18:07 +0000 X-ASF-Spam-Status: No, hits=-10.3 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_HI,SPF_PASS,USER_IN_DEF_SPF_WL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of prvs=402d680d9=leidle@amazon.com designates 207.171.184.25 as permitted sender) Received: from [207.171.184.25] (HELO smtp-fw-9101.amazon.com) (207.171.184.25) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Nov 2014 19:17:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1417202279; x=1448738279; h=from:to:subject:date:message-id:mime-version; bh=9XDBXA5OaCrM9zfH32D99nS2ltP1Zdi+FzhnIoW5Z5Y=; b=F0vqtbE29ipL1fom5xtwqWN+/KF+tLvP9x6gbryUUBeK2wLb/Ryd/3TA i8VLxEQJMIaaHUt7edHE9uGGtg3wveyL5k0CU11qLSvMErDNt7o0gGjYd g2BrWH8Rgk3RJaRofK1QrADTMvW2S3TJAeZ1j5WEWkPMQsmS2de7A0ar0 4=; X-IronPort-AV: E=Sophos;i="5.07,478,1413244800"; d="scan'208,217";a="148830599" Received: from email-inbound-relay-62005.pdx2.amazon.com ([10.242.45.184]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 28 Nov 2014 19:14:36 +0000 Received: from ex10-hub-9004.ant.amazon.com (pdx2-ws-svc-lb17-vlan2.amazon.com [10.247.140.66]) by email-inbound-relay-62005.pdx2.amazon.com (8.14.7/8.14.7) with ESMTP id sASJEaWK010022 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=OK) for ; Fri, 28 Nov 2014 19:14:36 GMT Received: from EX10-MBX-9003.ant.amazon.com ([fe80::b9e5:5388:f95f:c940]) by ex10-hub-9004.ant.amazon.com ([::1]) with mapi id 14.03.0181.006; Fri, 28 Nov 2014 11:14:36 -0800 From: "Leidle, Rob" To: "user@bigtop.apache.org" Subject: Problem using puppet scripts to configure bigtop on AmazonLinux Thread-Topic: Problem using puppet scripts to configure bigtop on AmazonLinux Thread-Index: AQHQCz+LSgtMW2sGkEyVizWUgVaJsw== Date: Fri, 28 Nov 2014 19:14:35 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.6.141106 x-originating-ip: [10.184.49.70] Content-Type: multipart/alternative; boundary="_000_D09E037E36B46leidleamazoncom_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_D09E037E36B46leidleamazoncom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello all, I am trying to configure & install Bigtop 0.8.0 using the puppet= scripts on AmazonLinux on EC2. Thus far, almost everything has worked besi= des one minor change I have made to the site.pp manifest. However, I am run= ning into a problem, it seems that the services such as proxy server or nam= enode are not immediately starting. You can see the error below in the purp= le text related to namenode. info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-yarn-resourcemanager] info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-mapreduce-historyserver] info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-yarn-nodemanager] info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-yarn-proxyserver] debug: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml= ]: The container Class[Hadoop::Common-yarn] will propagate my refresh event debug: Service[hadoop-yarn-proxyserver](provider=3Dredhat): Executing '/sbi= n/service hadoop-yarn-proxyserver status' debug: Service[hadoop-yarn-proxyserver](provider=3Dredhat): Executing '/sbi= n/service hadoop-yarn-proxyserver start' err: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service= [hadoop-yarn-proxyserver]/ensure: change from stopped to running failed: Co= uld not start Service[hadoop-yarn-proxyserver]: Execution of '/sbin/service= hadoop-yarn-proxyserver start' returned 3: at /mnt/var/lib/bootstrap-acti= ons/1/bigtop-0.8.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:48= 3 debug: Service[hadoop-yarn-proxyserver](provider=3Dredhat): Executing '/sbi= n/service hadoop-yarn-proxyserver status' debug: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Servi= ce[hadoop-yarn-proxyserver]: Skipping restart; service is not running notice: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Serv= ice[hadoop-yarn-proxyserver]: Triggered 'refresh' from 4 events debug: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Servi= ce[hadoop-yarn-proxyserver]: The container Hadoop::Proxyserver[proxyserver]= will propagate my refresh event debug: Hadoop::Proxyserver[proxyserver]: The container Class[Hadoop_head_no= de] will propagate my refresh event debug: Class[Hadoop::Common-yarn]: The container Stage[main] will propagate= my refresh event debug: Service[hadoop-hdfs-namenode](provider=3Dredhat): Executing '/sbin/s= ervice hadoop-hdfs-namenode status' debug: Service[hadoop-hdfs-namenode](provider=3Dredhat): Executing '/sbin/s= ervice hadoop-hdfs-namenode start' err: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoo= p-hdfs-namenode]/ensure: change from stopped to running failed: Could not s= tart Service[hadoop-hdfs-namenode]: Execution of '/sbin/service hadoop-hdfs= -namenode start' returned 3: at /mnt/var/lib/bootstrap-actions/1/bigtop-0.= 8.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:335 debug: Service[hadoop-hdfs-namenode](provider=3Dredhat): Executing '/sbin/s= ervice hadoop-hdfs-namenode status' debug: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[had= oop-hdfs-namenode]: Skipping restart; service is not running notice: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[ha= doop-hdfs-namenode]: Triggered 'refresh' from 4 events debug: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[had= oop-hdfs-namenode]: The container Hadoop::Namenode[namenode] will propagate= my refresh event debug: Hadoop::Namenode[namenode]: The container Class[Hadoop_head_node] wi= ll propagate my refresh event notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[= hadoop-hdfs-datanode]: Dependency Service[hadoop-hdfs-namenode] has failure= s: true warning: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package= [hadoop-hdfs-datanode]: Skipping because of failed dependencies notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mn= t1/hdfs]: Dependency Service[hadoop-hdfs-namenode] has failures: true warning: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/m= nt1/hdfs]: Skipping because of failed dependencies notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mn= t/hdfs]: Dependency Service[hadoop-hdfs-namenode] has failures: true The interesting part is that if I query namenode status eventually the name= node will show up as started even though I have not taken any other actions= : [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status Hadoop namenode is not running [FAILED] [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status Hadoop namenode is not running [FAILED] [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status Hadoop namenode is running [ OK ] The same goes for proxy server. The problem here is that all other dependen= cies of namenode do not install (such as resource manager, etc). I am using= the latest release of AmazonLinux (2014.09) and this has puppet 2.7.25-1. = I am not sure what to do about this issue, has anyone else experienced some= thing like this? Should I just move to puppet 3.x and only try to install o= ut of the Bigtop trunk (0.9.0)? --_000_D09E037E36B46leidleamazoncom_ Content-Type: text/html; charset="us-ascii" Content-ID: <8DA8B24ACDF88D47A9D7CED8B0221DFF@ant.amazon.com> Content-Transfer-Encoding: quoted-printable
Hello all, I am trying to configure & install Bigtop 0.8.0 using t= he puppet scripts on AmazonLinux on EC2. Thus far, almost everything has wo= rked besides one minor change I have made to the site.pp manifest. However,= I am running into a problem, it seems that the services such as proxy server or namenode are not immediately sta= rting. You can see the error below in the purple text related to namenode.<= /div>

info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-yarn-resourcemanager]

info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-mapreduce-historyserver]

info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-yarn-nodemanager]

info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]= : Scheduling refresh of Service[hadoop-yarn-proxyserver]

debug: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml= ]: The container Class[Hadoop::Common-yarn] will propagate my refresh event=

debug: Service[hadoop-yarn-proxyserver](provider=3Dredhat): Executing '/sbi= n/service hadoop-yarn-proxyserver status'

debug: Service[hadoop-yarn-proxyserver](provider=3Dredhat): Executing '/sbi= n/service hadoop-yarn-proxyserver start'

err: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service= [hadoop-yarn-proxyserver]/ensure: change from stopped to running failed: Co= uld not start Service[hadoop-yarn-proxyserver]: Execution of '/sbin/service= hadoop-yarn-proxyserver start' returned 3:  at /mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-= deploy/puppet/modules/hadoop/manifests/init.pp:483

debug: Service[hadoop-yarn-proxyserver](provider=3Dredhat): Executing '/sbi= n/service hadoop-yarn-proxyserver status'

debug: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Servi= ce[hadoop-yarn-proxyserver]: Skipping restart; service is not running

notice: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Serv= ice[hadoop-yarn-proxyserver]: Triggered 'refresh' from 4 events

debug: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Servi= ce[hadoop-yarn-proxyserver]: The container Hadoop::Proxyserver[proxyserver]= will propagate my refresh event

debug: Hadoop::Proxyserver[proxyserver]: The container Class[Hadoop_head_no= de] will propagate my refresh event

debug: Class[Hadoop::Common-yarn]: The container Stage[main] will propagate= my refresh event

debug: Service[hadoop-hdfs-namenode](provider=3Dredhat): Executing '/sbin/s= ervice hadoop-hdfs-namenode status'

debug: Service[hadoop-hdfs-namenode](provider=3Dredhat): Executing '/sbin/s= ervice hadoop-hdfs-namenode start'

err: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoo= p-hdfs-namenode]/ensure: change from stopped to running failed: Could not s= tart Service[hadoop-hdfs-namenode]: Execution of '/sbin/service hadoop-hdfs= -namenode start' returned 3:  at /mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-deploy/puppet/modules= /hadoop/manifests/init.pp:335

debug: Service[hadoop-hdfs-namenode](provider=3Dredhat): Executing '/sbin/s= ervice hadoop-hdfs-namenode status'

debug: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[had= oop-hdfs-namenode]: Skipping restart; service is not running

notice: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[ha= doop-hdfs-namenode]: Triggered 'refresh' from 4 events

debug: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[had= oop-hdfs-namenode]: The container Hadoop::Namenode[namenode] will propagate= my refresh event

debug: Hadoop::Namenode[namenode]: The container Class[Hadoop_head_node] wi= ll propagate my refresh event

notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[= hadoop-hdfs-datanode]: Dependency Service[hadoop-hdfs-namenode] has failure= s: true

warning: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package= [hadoop-hdfs-datanode]: Skipping because of failed dependencies

notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mn= t1/hdfs]: Dependency Service[hadoop-hdfs-namenode] has failures: true

warning: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/m= nt1/hdfs]: Skipping because of failed dependencies

notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mn= t/hdfs]: Dependency Service[hadoop-hdfs-namenode] has failures: true



The interesting part is that if I query namenode status eventually the= namenode will show up as started even though I have not taken any other ac= tions:

[hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status=

Hadoop namenode is not running             &n= bsp;               [FAILED]

[hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status=

Hadoop namenode is not running             &n= bsp;               [FAILED]

[hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status=

Hadoop namenode is running              =                   [  OK=   ]


The same goes for proxy server. The problem here is that all other dep= endencies of namenode do not install (such as resource manager, etc). I am = using the latest release of AmazonLinux (2014.09) and this has puppet 2.7.2= 5-1. I am not sure what to do about this issue, has anyone else experienced something like this? Should I just= move to puppet 3.x and only try to install out of the Bigtop trunk (0.9.0)= ?
--_000_D09E037E36B46leidleamazoncom_--