hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Krogen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13747) Use LongAdder for more efficient metrics tracking
Date Fri, 21 Oct 2016 19:59:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596187#comment-15596187
] 

Erik Krogen commented on HADOOP-13747:
--------------------------------------

Hi [~stevel@apache.org], thanks for the comment. We are aware of existing built-in / library
implementations of {{LongAdder}} and aren't proposing implementing our own, just using them
to reimplement some of the existing metrics types to be more performant under high contention.
On Java 8 we would use the built-in and on Java 7 we would use the Guava or codahale versions.


It seems that the existing usages of codahale within the Hadoop code base sidestep the standard
Hadoop metrics system. Is that something we want to encourage? I was under the impression
that we should be standardizing on the Hadoop metrics system. I think fully adopting codahale
is a larger scale project than we want to address in this ticket. Please correct me if I am
wrong anywhere here. 

> Use LongAdder for more efficient metrics tracking
> -------------------------------------------------
>
>                 Key: HADOOP-13747
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13747
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Zhe Zhang
>            Assignee: Erik Krogen
>
> Currently many metrics, including {{RpcMetrics}} and {{RpcDetailedMetrics}}, use a synchronized
counter to be updated by all handler threads (multiple hundreds in large production clusters).
As [~andrew.wang] suggested, it'd be more efficient to use the [LongAdder | http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/jsr166e/LongAdder.java?view=co]
library which dynamically create intermediate-result variables.
> Assigning to [~xkrogen] who has already done some investigation on this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message