incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Talbot <btal...@aeriagames.com>
Subject MBean cassandra.db.CompactionManager TotalBytesCompacted counts backwards
Date Fri, 05 Oct 2012 23:27:13 GMT
I've recently added compaction rate (in bytes / second) to my monitors for
cassandra and am seeing some odd values.  I wasn't expecting the values for
TotalBytesCompacted to sometimes decrease from one reading to the next.  It
seems that the value should be monotonically increasing while a server is
running -- obviously it would start again at 0 when the server is restarted
or if the counter rolls over (unlikely for a 64 bit long).

Below are two samples taken 60 seconds apart: the value decreased by
2,954,369,012 between the two readings.

reported_metric=[timestamp:1349476449, status:200,
request:[mbean:org.apache.cassandra.db:type=CompactionManager,
attribute:TotalBytesCompacted, type:read], value:7548675470069]

previous_metric=[timestamp:1349476389, status:200,
request:[mbean:org.apache.cassandra.db:type=CompactionManager,
attribute:TotalBytesCompacted, type:read], value:7551629839081]


I briefly looked at the code for CompactionManager and a few related
classes and don't see anyplace that is performing subtraction explicitly;
however, there are many additions of signed long values that are not
validated and could conceivably contain a negative value thus causing the
totalBytesCompacted to decrease.  It's interesting to note that the all of
the differences I've seen so far are more than the overflow value of a
signed 32 bit value.  The OS (CentOS 5.7) and sun java vm (1.6.0_29) are
both 64 bit.  JNA is enabled.

Is this expected and normal?  If so, what is the correct interpretation of
this metric?  I'm seeing the negatives values a few times per hour when
reading it once every 60 seconds.

-Bryan

Mime
View raw message