ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddharth Wagle <swa...@hortonworks.com>
Subject Re: Kafka broker metrics not appearing in REST API
Date Thu, 07 May 2015 03:30:00 GMT
Hi Jayesh,


There is little to go by as far as debugging info here since based on what you observed host
and cluster metrics are being reported.


Could you take a look at Kafka logs and check for exception if any?


BR,

Sid



________________________________
From: Jayesh Thakrar <j_thakrar@yahoo.com>
Sent: Wednesday, May 06, 2015 8:07 PM
To: Siddharth Wagle; user@ambari.apache.org
Subject: Re: Kafka broker metrics not appearing in REST API

Hi Siddharth,

Yes, I am using Ambari 2.0 with Ambari Metrics service.
The interesting thing is that I got them for some time and not anymore.
And I also know that the metrics are being collected since i can see them on the dashboard.
Any pointer for troubleshooting?

And btw, it would be nice to have a count of messages received and not a computed metric count
/ min.
TSDB does a good job of giving me cumulative and rate-per-sec graphs and numbers.

Thanks in advance,
Jayesh

________________________________
From: Siddharth Wagle <swagle@hortonworks.com>
To: "user@ambari.apache.org" <user@ambari.apache.org>; Jayesh Thakrar <j_thakrar@yahoo.com>
Sent: Wednesday, May 6, 2015 10:03 PM
Subject: Re: Kafka broker metrics not appearing in REST API

Hi Jayesh,

Are you using Ambari 2.0 with Ambari Metrics service?

BR,
Sid


________________________________
From: Jayesh Thakrar <j_thakrar@yahoo.com>
Sent: Wednesday, May 06, 2015 7:53 PM
To: user@ambari.apache.org
Subject: Kafka broker metrics not appearing in REST API

Hi,

I have installed 2 clusters with Ambari and Storm and Kafka.
After the install, I was able to get metrics for both Storm and Kafka via REST API.
This worked fine for a week, but since the past 2 days, I have not been getting Kafka metrics.

I need the metrics to push to an OpenTSDB cluster.
I do get host metrics and Nimbus metrics but not KAFKA_BROKER metrics.

I did have maintenance turned on for some time, but maintenance is turned off now.

[jthakrar@dtord01hdp0101d ~]$ curl --user admin:admin 'http://dtord01flm01p:8080/api/v1/clusters/ord_flume_kafka_prod/components/NIMBUS?fields=metrics'
{
  "href" : "http://dtord01flm01p:8080/api/v1/clusters/ord_flume_kafka_prod/components/NIMBUS?fields=metrics",
  "ServiceComponentInfo" : {
    "cluster_name" : "ord_flume_kafka_prod",
    "component_name" : "NIMBUS",
    "service_name" : "STORM"
  },
  "metrics" : {
    "storm" : {
      "nimbus" : {
        "freeslots" : 54.0,
        "supervisors" : 27.0,
        "topologies" : 0.0,
        "totalexecutors" : 0.0,
        "totalslots" : 54.0,
        "totaltasks" : 0.0,
        "usedslots" : 0.0
      }
    }
  }
}

[jthakrar@dtord01hdp0101d ~]$ curl --user admin:admin 'http://dtord01flm01p:8080/api/v1/clusters/ord_flume_kafka_prod/components/KAFKA_BROKER?fields=metrics'
{
  "href" : "http://dtord01flm01p:8080/api/v1/clusters/ord_flume_kafka_prod/components/KAFKA_BROKER?fields=metrics",
  "ServiceComponentInfo" : {
    "cluster_name" : "ord_flume_kafka_prod",
    "component_name" : "KAFKA_BROKER",
    "service_name" : "KAFKA"
  }
}

[jthakrar@dtord01hdp0101d ~]$ curl --user admin:admin 'http://dtord01flm01p:8080/api/v1/clusters/ord_flume_kafka_prod/components/SUPERVISOR?fields=metrics'
{
  "href" : "http://dtord01flm01p:8080/api/v1/clusters/ord_flume_kafka_prod/components/SUPERVISOR?fields=metrics",
  "ServiceComponentInfo" : {
    "cluster_name" : "ord_flume_kafka_prod",
    "component_name" : "SUPERVISOR",
    "service_name" : "STORM"
  }




Mime
View raw message