kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Swapnil Ghike (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1100) metrics shouldn't have generation/timestamp specific names
Date Wed, 23 Oct 2013 06:27:43 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13802635#comment-13802635

Swapnil Ghike commented on KAFKA-1100:

Hi Jason, at LinkedIn, we use wildcards/regexes to create graphs from such mbeans. Would you
be able to do something similar?

> metrics shouldn't have generation/timestamp specific names
> ----------------------------------------------------------
>                 Key: KAFKA-1100
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1100
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Rosenberg
> I've noticed that there are several metrics that seem useful for monitoring overtime,
but which contain generational timestamps in the metric name.
> We are using yammer metrics libraries to send metrics data in a background thread every
10 seconds (to kafka actually), and then they eventually end up in a metrics database (graphite,
opentsdb).  The metrics then get graphed via UI, and we can see metrics going way back, etc.
> Unfortunately, many of the metrics coming from kafka seem to have metric names that change
any time the server or consumer is restarted, which makes it hard to easily create graphs
over long periods of time (spanning app restarts).
> For example:
> names like: 
> kafka.consumer.FetchRequestAndResponseMetrics....square-1371718712833-e9bb4d10-0-508818741-AllBrokersFetchRequestRateAndTimeMs
> or: 
> kafka.consumer.ZookeeperConsumerConnector...topicName.....square-1373476779391-78aa2e83-0-FetchQueueSize
> In our staging environment, we have our servers on regular auto-deploy cycles (they restart
every few hours).  So just not longitudinally usable to have metric names constantly changing
like this.
> Is there something that can easily be done?  Is it really necessary to have so much cryptic
info in the metric name?

This message was sent by Atlassian JIRA

View raw message