hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10839) Add unregisterSource() to MetricsSystem API
Date Thu, 17 Jul 2014 10:38:11 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064795#comment-14064795
] 

Hudson commented on HADOOP-10839:
---------------------------------

FAILURE: Integrated in Hadoop-Yarn-trunk #615 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/615/])
HADOOP-10839. Add unregisterSource() to MetricsSystem API. Contributed by Shanyu Zhao. (cnauroth:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611134)
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestMetricsSystemImpl.java


> Add unregisterSource() to MetricsSystem API
> -------------------------------------------
>
>                 Key: HADOOP-10839
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10839
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>    Affects Versions: 2.4.1
>            Reporter: shanyu zhao
>            Assignee: shanyu zhao
>             Fix For: 3.0.0, 2.6.0
>
>         Attachments: HADOOP-10839.2.patch, HADOOP-10839.patch
>
>
> Currently the MetrisSystem API has register() method to register a MetricsSource but
doesn't have unregister() method. This means once a MetricsSource is registered with the MetricsSystem,
it will be there forever until the MetricsSystem is shut down. This in some cases can cause
Java OutOfMemoryError.
> One such case is in file system metrics implementation. The new AbstractFileSystem/FileContext
framework does not implement a cache so every file system access can lead to the creation
of a NativeFileSystem instance. (refer to HADOOP-6356). And all these NativeFileSystem needs
to share the same instance of MetricsSystemImpl, which means we cannot shut down MetricsSystem
to clean up all the MetricsSources that has been registered but no longer active. Over time
the MetricsSource instance accumulates and eventually we saw OutOfMemoryError.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message