hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Krogen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13782) Make MutableRates metrics thread-local write, aggregate-on-read
Date Mon, 07 Nov 2016 23:05:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15645764#comment-15645764

Erik Krogen commented on HADOOP-13782:

[~zhz], thanks for the review.

1. Yes, it would be ideal. I've opened HADOOP-13804 to address the issue with {{MutableStat}}
which was causing it to be unfit to use here. Fixing this issue will enable us to remove both
inner classes. 
2. We want to ensure that {{add}} is a non-synchronized method to allow for concurrency (otherwise
we would defeat the advantages of this class over {{MutableRates}}). A snapshot will not present
a point-in-time consistent snapshot across all threads since that would require a higher degree
of synchronization, but it does not seem that this should be necessary given that the values
are just metrics and the only inexactness is exactly which side of a snapshot a metric's addition
appears on (i.e. no values will be lost). 
3. Sure.
4. {{getLocalMetrics}} was originally used in two places which is why I had it has a separate
method. Good catch that this is no longer necessary. 
5. Agreed that this would be ideal. 

I'm attaching a new patch, v002, addressing these changes but unmarking patch available since
these changes rely on the patch in HADOOP-13804. 

> Make MutableRates metrics thread-local write, aggregate-on-read
> ---------------------------------------------------------------
>                 Key: HADOOP-13782
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13782
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>         Attachments: HADOOP-13782.000.patch, HADOOP-13782.001.patch
> Currently the {{MutableRates}} metrics class serializes all writes to metrics it contains
because of its use of {{MetricsRegistry.add()}} (i.e., even two increments of unrelated metrics
contained within the same {{MutableRates}} object will serialize w.r.t. each other). This
class is used by {{RpcDetailedMetrics}}, which may have many hundreds of threads contending
to modify these metrics. Instead we should allow updates to unrelated metrics objects to happen
concurrently. To do so we can let each thread locally collect metrics, and on a {{snapshot}},
aggregate the metrics from all of the threads. 
> I have collected some benchmark performance numbers in HADOOP-13747 (https://issues.apache.org/jira/secure/attachment/12835043/benchmark_results)
which indicate that this can bring significantly higher performance in high contention situations.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message