brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duncan Godwin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BROOKLYN-319) RabbitMQ restart failure
Date Thu, 28 Jul 2016 14:11:20 GMT
Duncan Godwin created BROOKLYN-319:
--------------------------------------

             Summary: RabbitMQ restart failure
                 Key: BROOKLYN-319
                 URL: https://issues.apache.org/jira/browse/BROOKLYN-319
             Project: Brooklyn
          Issue Type: Bug
         Environment: Centos 6 on BlueBox and Google Compute
            Reporter: Duncan Godwin


RabbitMQ intermittently fails to come back up after restart of the entity on some clouds (BlueBox
and Google Compute known to fail).

The brooklyn.debug.log shows the following:

{code}
2016-07-28 14:05:12,642 DEBUG brooklyn.SSH [brooklyn-execmanager-E26Om0Sg-12886]: check-running
RabbitBrokerImpl{id=d5kw04uk46, amqpPort=5672}, initiating ssh on machine SshMachineLocation[10.104.2.181:amp@10.104.2.181/10.104.2.181:22(id=t0tvrv6o0h)]
(env {RABBITMQ_HOME=/home/users/amp/brooklyn-managed-processes/apps/b6kxbm5fei/entities/RabbitBroker_d5kw04uk46,
RABBITMQ_LOG_BASE=/home/users/amp/brooklyn-managed-processes/apps/b6kxbm5fei/entities/RabbitBroker_d5kw04uk46,
RABBITMQ_NODENAME=d5kw04uk46, RABBITMQ_NODE_PORT=5672, RABBITMQ_PID_FILE=/home/users/amp/brooklyn-managed-processes/apps/b6kxbm5fei/entities/RabbitBroker_d5kw04uk46/rabbitmq.pid,
RABBITMQ_CONFIG_FILE=/home/users/amp/brooklyn-managed-processes/apps/b6kxbm5fei/entities/RabbitBroker_d5kw04uk46/rabbitmq}):
#!/bin/bash -e
 ; export INSTALL_DIR="/home/users/amp/brooklyn-managed-processes/installs/RabbitBroker_3.6.0"
; export RUN_DIR="/home/users/amp/brooklyn-managed-processes/apps/b6kxbm5fei/entities/RabbitBroker_d5kw04uk46"
; mkdir -p $RUN_DIR ; cd $RUN_DIR ; ./sbin/rabbitmqctl -q status
2016-07-28 14:05:12,982 DEBUG o.a.b.u.c.i.w.w.Winrm4jTool [brooklyn-execmanager-E26Om0Sg-12879]:
Finished WinRM exec on Administrator@217.115.71.201:11049  done after 2.01s (connected in
0ms)
2016-07-28 14:05:13,012 DEBUG brooklyn.SSH [Thread-350446]: [d5kw04uk46@10.104.2.181:stdout]
Executed /tmp/brooklyn-20160728-140512642-NDCx-check-running_RabbitBrokerImpl.sh, result 69
2016-07-28 14:05:13,012 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
Error: unable to connect to node 'd5kw04uk46@qa-scheduler-rabbitmq-server-be5': nodedown
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
DIAGNOSTICS
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
===========
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
attempted to contact: ['d5kw04uk46@qa-scheduler-rabbitmq-server-be5']
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
d5kw04uk46@qa-scheduler-rabbitmq-server-be5:
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
  * connected to epmd (port 4369) on qa-scheduler-rabbitmq-server-be5
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
  * epmd reports: node 'd5kw04uk46' not running at all
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
                  no other nodes on qa-scheduler-rabbitmq-server-be5
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
  * suggestion: start the node
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
current node details:
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
- node name: 'rabbitmq-cli-23557@qa-scheduler-rabbitmq-server-be5'
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
- home dir: /home/users/amp
2016-07-28 14:05:13,013 DEBUG brooklyn.SSH [Thread-350447]: [d5kw04uk46@10.104.2.181:stderr]
- cookie hash: hGw+myYNiff8VA9lwwfh0g==
{code}

the console-out.log contains:

{code}
ERROR: node with name "d5kw04uk46" already running on "qa-scheduler-rabbitmq-server-be5"
{code}

The RabbitMQ service is not runing on the machine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message