cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain Rastoul <alf.mmm....@gmail.com>
Subject Re: weird jvm metrics
Date Thu, 05 Jan 2017 18:53:59 GMT
On 01/04/2017 11:12 PM, Edward Capriolo wrote:
> The metric-reporter is actually leveraged from another project.
>
> https://github.com/addthis/metrics-reporter-config
>
> Check the version of metric-reporter (in cassandra/lib) and see if it
> has changed from your old version to your new version.
>
>
>
>
> On Wed, Jan 4, 2017 at 12:02 PM, Mike Torra <mtorra@demandware.com
> <mailto:mtorra@demandware.com>> wrote:
>
>     Just bumping - has anyone seen this before?
>
>     http://stackoverflow.com/questions/41446352/cassandra-3-9-jvm-metrics-have-bad-name
>     <http://stackoverflow.com/questions/41446352/cassandra-3-9-jvm-metrics-have-bad-name>
>
>     From: Mike Torra <mtorra@demandware.com <mailto:mtorra@demandware.com>>
>     Reply-To: "user@cassandra.apache.org
>     <mailto:user@cassandra.apache.org>" <user@cassandra.apache.org
>     <mailto:user@cassandra.apache.org>>
>     Date: Wednesday, December 28, 2016 at 4:49 PM
>     To: "user@cassandra.apache.org <mailto:user@cassandra.apache.org>"
>     <user@cassandra.apache.org <mailto:user@cassandra.apache.org>>
>     Subject: weird jvm metrics
>
>     Hi There -
>
>     I recently upgraded from cassandra 3.5 to 3.9 (DDC), and I noticed
>     that the "new" jvm metrics are reporting with an extra '.' character
>     in them. Here is a snippet of what I see from one of my nodes:
>
>     ubuntu@ip-10-0-2-163:~$ sudo tcpdump -i eth0 -v dst port 2003 -A |
>     grep 'jvm'
>
>     tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture
>     size 65535 bytes
>
>     .Je..l>.pi.cassandra.us-east-1.cassy-node1.jvm.buffers..direct.capacity
>     762371494 1482960946
>
>     pi.cassandra.us-east-1.cassy-node1.jvm.buffers..direct.count 3054
>     1482960946
>
>     pi.cassandra.us-east-1.cassy-node1.jvm.buffers..direct.used
>     762371496 1482960946
>
>     pi.cassandra.us-east-1.cassy-node1.jvm.buffers..mapped.capacity
>     515226631134 1482960946
>
>     pi.cassandra.us-east-1.cassy-node1.jvm.buffers..mapped.count 45572
>     1482960946
>
>     pi.cassandra.us-east-1.cassy-node1.jvm.buffers..mapped.used
>     515319762610 1482960946
>
>     pi.cassandra.us-east-1.cassy-node1.jvm.fd.usage 0.00 1482960946
>
>
>     My metrics.yaml looks like this:
>
>     graphite:
>        -
>          period: 60
>          timeunit: 'SECONDS'
>          prefix: 'pi.cassandra.us-east-1.cassy-node1'
>          hosts:
>           - host: '#RELAY_HOST#'
>             port: 2003
>          predicate:
>            color: "white"
>            useQualifiedName: true
>            patterns:
>              - "^org.+"
>              - "^jvm.+"
>              - "^java.lang.+"
>
>     All the org.* metrics come through fine, and the jvm.fd.usage metric
>     strangely comes through fine, too. The rest of the jvm.* metrics
>     have this extra '.' character that causes them to not show up in
>     graphite.
>
>     Am I missing something silly here? Appreciate any help or suggestions.
>
>     - Mike
>
>
Hi,

I also noticed this problem recently.
Some jvm metrics have a double dot in name like:
jvm.memory..total.max , jvm.memory..total.init (etc).
Investigating a bit, it seems that an extra dot is added at the end of 
the name in CassandraDaemon.java, around line 367 (in 3.0.10):
...
                 // enable metrics provided by metrics-jvm.jar
 
CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new 
BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer()));
                 CassandraMetricsRegistry.Metrics.register("jvm.gc.", 
new GarbageCollectorMetricSet());
 
CassandraMetricsRegistry.Metrics.register("jvm.memory.", new 
MemoryUsageGaugeSet());

and also added in append method of MetricRegistry.
Call stack is:
MetricRegistry>>registerAll(String prefix, MetricSet metrics)
MetricRegistry>>static String name(String name, String... names)
MetricRegistry>>static void append(StringBuilder builder, String part)

in append the dot is also added:
...
             if(builder.length() > 0) {
                 builder.append('.');
             }
             builder.append(part);
...

The codahale MetricRegistry class seems to have no recent modification 
of name or append methods, so it looks like a small (but annoying) bug.
May be the fix could be to simply remove the final dot ?

Is it worth opening an issue in JIRA ?


-- 
best,
Alain


Mime
View raw message