bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Problem using puppet scripts to configure bigtop on AmazonLinux
Date Sat, 29 Nov 2014 00:14:29 GMT
On Fri, Nov 28, 2014 at 08:06PM, Leidle, Rob wrote:
> Hello all, I am trying to configure & install Bigtop 0.8.0 using the puppet scripts
on AmazonLinux on EC2. Thus far, almost everything has worked besides one minor change I have
made to the site.pp manifest. However, I am running into a problem, it seems that the services
such as proxy server or namenode are not immediately starting. You can see the error below
in the purple text related to namenode.
> 
> 
> info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: Scheduling
refresh of Service[hadoop-yarn-resourcemanager]
> 
> info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: Scheduling
refresh of Service[hadoop-mapreduce-historyserver]
> 
> info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: Scheduling
refresh of Service[hadoop-yarn-nodemanager]
> 
> info: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: Scheduling
refresh of Service[hadoop-yarn-proxyserver]
> 
> debug: /Stage[main]/Hadoop::Common-yarn/File[/etc/hadoop/conf/yarn-site.xml]: The container
Class[Hadoop::Common-yarn] will propagate my refresh event
> 
> debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing '/sbin/service hadoop-yarn-proxyserver
status'
> 
> debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing '/sbin/service hadoop-yarn-proxyserver
start'
> 
> err: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]/ensure:
change from stopped to running failed: Could not start Service[hadoop-yarn-proxyserver]: Execution
of '/sbin/service hadoop-yarn-proxyserver start' returned 3:  at /mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:483
> 
> debug: Service[hadoop-yarn-proxyserver](provider=redhat): Executing '/sbin/service hadoop-yarn-proxyserver
status'
> 
> debug: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]:
Skipping restart; service is not running
> 
> notice: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]:
Triggered 'refresh' from 4 events
> 
> debug: /Stage[main]/Hadoop_head_node/Hadoop::Proxyserver[proxyserver]/Service[hadoop-yarn-proxyserver]:
The container Hadoop::Proxyserver[proxyserver] will propagate my refresh event
> 
> debug: Hadoop::Proxyserver[proxyserver]: The container Class[Hadoop_head_node] will propagate
my refresh event
> 
> debug: Class[Hadoop::Common-yarn]: The container Stage[main] will propagate my refresh
event
> 
> debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing '/sbin/service hadoop-hdfs-namenode
status'
> 
> debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing '/sbin/service hadoop-hdfs-namenode
start'
> 
> err: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]/ensure:
change from stopped to running failed: Could not start Service[hadoop-hdfs-namenode]: Execution
of '/sbin/service hadoop-hdfs-namenode start' returned 3:  at /mnt/var/lib/bootstrap-actions/1/bigtop-0.8.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:335
> 
> debug: Service[hadoop-hdfs-namenode](provider=redhat): Executing '/sbin/service hadoop-hdfs-namenode
status'
> 
> debug: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]:
Skipping restart; service is not running
> 
> notice: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]:
Triggered 'refresh' from 4 events
> 
> debug: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Service[hadoop-hdfs-namenode]:
The container Hadoop::Namenode[namenode] will propagate my refresh event
> 
> debug: Hadoop::Namenode[namenode]: The container Class[Hadoop_head_node] will propagate
my refresh event
> 
> notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[hadoop-hdfs-datanode]:
Dependency Service[hadoop-hdfs-namenode] has failures: true
> 
> warning: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Package[hadoop-hdfs-datanode]:
Skipping because of failed dependencies
> 
> notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt1/hdfs]:
Dependency Service[hadoop-hdfs-namenode] has failures: true
> 
> warning: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt1/hdfs]:
Skipping because of failed dependencies
> 
> notice: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/File[/mnt/hdfs]: Dependency
Service[hadoop-hdfs-namenode] has failures: true
> 
> 
> The interesting part is that if I query namenode status eventually the namenode will
show up as started even though I have not taken any other actions:
> 
> 
> [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status
> 
> Hadoop namenode is not running                             [FAILED]
> 
> [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status
> 
> Hadoop namenode is not running                             [FAILED]
> 
> [hadoop@ip-10-168-87-216 1]$ sudo /sbin/service hadoop-hdfs-namenode status
> 
> Hadoop namenode is running                                 [  OK  ]
> 
> The same goes for proxy server. The problem here is that all other
> dependencies of namenode do not install (such as resource manager, etc). I
> am using the latest release of AmazonLinux (2014.09) and this has puppet
> 2.7.25-1. I am not sure what to do about this issue, has anyone else
> experienced something like this? Should I just move to puppet 3.x and only
> try to install out of the Bigtop trunk (0.9.0)?

ResourceManager isn't a dependency of namenode - it's the other way around.
It's hard to say what's going on with your system without looking into
particular daemon logs. I'd suggest you check them and investigate what's the
trouble is. 

Also, there's a small issue BIGTOP-1522 with nodemanager recipes if you're
installing a custom set of components, which might or not affect you 

Cos


Mime
View raw message