cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Burman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-14281) Improve LatencyMetrics performance by reducing write path processing
Date Fri, 09 Mar 2018 09:45:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392655#comment-16392655
] 

Michael Burman commented on CASSANDRA-14281:
--------------------------------------------

Additional note.. there's also same kind of unnecessary multiple updates in the read path.
Including TableMetrics & TableHistogram's update. This patch obviously helps a little
bit, since it reduces the histogram update time, but there's most likely stuff to reduce.
Fixing those helps in the write path the: "metric.colUpdateTimeDeltaHistogram.update(Math.min(18165375903306L,
timeDelta));" line, but these are very minor performance issues in the update path at the
moment. Another ticket for read path?

> Improve LatencyMetrics performance by reducing write path processing
> --------------------------------------------------------------------
>
>                 Key: CASSANDRA-14281
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14281
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Michael Burman
>            Assignee: Michael Burman
>            Priority: Major
>
> Currently for each write/read/rangequery/CAS touching the CFS we write a latency metric
which takes a lot of processing time (up to 66% of the total processing time if the update
was empty). 
> The way latencies are recorded is to use both a dropwizard "Timer" as well as "Counter".
Latter is used for totalLatency and the previous is decaying metric for rates and certain
percentile metrics. We then replicate all of these CFS writes to the KeyspaceMetrics and globalWriteLatencies. 
> Instead of doing this on the write phase we should merge the metrics when they're read.
This is much less common occurrence and thus we save a lot of CPU time in total. This also
speeds up the write path.
> Currently, the DecayingEstimatedHistogramReservoir acquires a lock for each update operation,
which causes a contention if there are more than one thread updating the histogram. This impacts
scalability when using larger machines. We should make it lock-free as much as possible and
also avoid a single CAS-update from blocking all the concurrent threads from making an update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message