hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maysam Yabandeh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11301) [optionally] update jmx cache to drop old metrics
Date Sat, 22 Nov 2014 00:21:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221637#comment-14221637
] 

Maysam Yabandeh commented on HADOOP-11301:
------------------------------------------

Sure. I have to admit the the logic behind updateJmxCache is a bit confusing for me and it
is not very clear for me that when the cache was intended to be updated. To reconstruct the
expected contract, which is missing in the code, one would need to review all the usages inside
the hadoop project. It seems that jmx meant to update the metrics with a frequency less then
or equal to 2*jmxCacheTTL.

As you suggested I will upload a patch with the conservative approach of keeping the current
implementation backward compatible with previous unit tests. 

> [optionally] update jmx cache to drop old metrics
> -------------------------------------------------
>
>                 Key: HADOOP-11301
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11301
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Maysam Yabandeh
>            Assignee: Maysam Yabandeh
>         Attachments: HADOOP-11301.v01.patch, HADOOP-11301.v02.patch, HADOOP-11301.v03.patch
>
>
> MetricsSourceAdapter::updateJmxCache() skips updating the info cache if no new metric
is added since last time:
> {code}
>       int oldCacheSize = attrCache.size();
>       int newCacheSize = updateAttrCache();
>       if (oldCacheSize < newCacheSize) {
>         updateInfoCache();
>       }
> {code}
> This behavior is not desirable in some applications. For example nntop (HDFS-6982) reports
the top users via jmx. The list is updated after each report. The previously reported top
users hence should be removed from the cache upon each report request.
> In our production run of nntop we made a change to ignore the size check and always perform
updateInfoCache. I am planning to submit a patch including this change. The feature can be
enabled by a configuration parameter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message