cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Per Otterström (JIRA) <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12876) Negative mean write latency
Date Thu, 10 Nov 2016 21:04:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15655141#comment-15655141
] 

Per Otterström commented on CASSANDRA-12876:
--------------------------------------------

Thanks for the input @alprema!

So, it appears that overflow doesn't happen during rescale, but when mean value is calculated.
Triggered when all bucket values are added together. The sudden stop of negative values happened
when rescale was triggerd.

The attached patch will perform a rescale of the snapshot values when the snapshot is created,
thereby avoiding overflow. As a side effect, this will make the max value to behave better.
Before max value would continue to raise and then get a sudden drop every 30 minutes when
rescale happend. Now, max value will appear to decay on a minute basis, whether value is peaking
just before or after rescale.

The patch includes a missing part from CASSANDRA-11752 that never were merged into 2.2 branch.
It seem to apply clean on 3.0.x as well.

> Negative mean write latency
> ---------------------------
>
>                 Key: CASSANDRA-12876
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12876
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Observability
>            Reporter: Kévin LOVATO
>            Assignee: Per Otterström
>         Attachments: 12876-2.2.txt, negative_mean.png, negative_mean_details.PNG, negative_mean_periodicity.PNG
>
>
> The mean write latency returned by JMX turns negative every 30 minutes. As the attached
screenshots show, the value turns negative every 30 minutes after the startup of the node.
> We did not experience this behavior in 2.1.16.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message