ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Mohanty <smoha...@hortonworks.com>
Subject Re: No Heartbeat after upgrade to Ambari 2.0
Date Fri, 17 Apr 2015 15:50:06 GMT
Can you share the output of

curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/hosts
curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://localhost:8080/api/v1/clusters/c1/hosts

Generic pattern of the calls being
curl -u <username>:<password> -H "X-Requested-By: ambari" -X GET http://<ambari_server_host>:8080/api/v1/clusters/<cluster_name>/hosts

perhaps there is a mismatch between hostnames between when the agents registered initially
vs when they register now.
________________________________________
From: Frank Eisenhauer <feisenhauer2@gmail.com>
Sent: Thursday, April 16, 2015 10:51 PM
To: user@ambari.apache.org
Subject: Re: No Heartbeat after upgrade to Ambari 2.0

Hi Sumit,
I stopped ambari agent, cleared the logs and started the agent again.
The log seems to be ok, there is only one warning which might be related
to the issue:

WARNING 2015-04-17 07:50:47,784 AlertSchedulerHandler.py:92 - There are
no alert definition commands in the heartbeat; unable to update definitions

There are entries in the log which seem to me as if the heartbeat was
sent to ambari server:
DEBUG 2015-04-17 07:50:57,796 Heartbeat.py:78 - Heartbeat:
{'componentStatus': [],
  'hostname': 'srv233.xxxxx.xxx',
  'nodeStatus': {'cause': 'NONE', 'status': 'HEALTHY'},
  'reports': [],
  'responseId': 0,
  'timestamp': 1429249857795}

We experienced some problems with upper case hostnames in the past.
Might that be a problem?

If you need more information from the log file, I'll look for a way to
share the log.


Am 17.04.2015 um 00:19 schrieb Sumit Mohanty:
> Those do not seem related to the error. Try this:
>
> You can pick one agent for which ambari-server is reporting Heartbeat lost.
>
> * Set the agent log to DEBUG (edit /etc/ambari-agent/conf/ambari-agent.ini)
> * Stop the agent and backup its log file
> * Start the agent and let it run for ~2 minutes
>
> Can you share the log through some public share? I think Apache strips off all attachments.
> ________________________________________
> From: Frank Eisenhauer <feisenhauer2@gmail.com>
> Sent: Thursday, April 16, 2015 3:06 PM
> To: user@ambari.apache.org
> Subject: Re: No Heartbeat after upgrade to Ambari 2.0
>
> Already checked them.
> There are entries like:
> INFO [pool-1-thread-5473] URLStreamProvider:144 - Received
> WWW-Authentication header:Negotiate, for
> URL:http://<host>:8744/api/v1/cluster/summary
> ERROR [pool-1-thread-5473] AppCookieManager:122 - SPNego authentication
> failed, can not get hadoop.auth cookie for URL:
> http://<host>:8744/api/v1/cluster
> ERROR [pool-1-thread-5473] AppCookieManager:122 - SPNego authentication
> failed, can not get hadoop.auth cookie for URL:
> http://<host>:8744/api/v1/cluster
>
> Am 16.04.2015 um 23:47 schrieb Sumit Mohanty:
>> Can you check ambari-agent logs (/var/log/ambari-agent/ambari-agent.log or /var/log/ambari-agent/ambari-agent.out)
and ambari-server logs (/var/log/ambari-server/ambari-server.log)?
>> ________________________________________
>> From: Frank Eisenhauer <feisenhauer2@gmail.com>
>> Sent: Thursday, April 16, 2015 2:32 PM
>> To: Ambari User
>> Subject: No Heartbeat after upgrade to Ambari 2.0
>>
>> Hi All,
>>
>> I've just upgraded my testcluster (3 Nodes) to Ambari 2.0 according to
>> the upgrade documentation of hortonworts.
>> The upgrade went without failures.
>> But after logging into Ambari Web, all Services show "Heartbeat lost".
>> I already restarted ambari agents on each host but status remains
>> "Heartbeat lost".
>>
>> Has anyone encoutered a similar result of the upgrade?

Mime
View raw message