hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HADOOP-6843) Entries in the FileSystem's Cache could be cleared when they are not used
Date Thu, 28 Apr 2011 22:42:03 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsz Wo (Nicholas), SZE resolved HADOOP-6843.
--------------------------------------------

    Resolution: Not A Problem

This is not a problem anymore.

> Entries in the FileSystem's Cache could be cleared when they are not used
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-6843
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6843
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 0.20.0
>            Reporter: Devaraj Das
>         Attachments: TestFileSystemCache.java, fs-weak-ref.4.patch
>
>
> In FileSystem, there is a cache maintained for Filesystem instances. The entries in the
cache are cleared only when explicit FileSystem.close is invoked. Applications are not careful
on this aspect. Typically, they do FileSystem.get(), operate on the FileSystem, and then they
just forget about it. Every FileSystem instance stores a reference to the Configuration object
that it was created with. Over a period of time, as the cache grows, this can lead to OOM
(we have seen this happening in our hadoop 20S clusters at Yahoo).
> This jira aims at addressing the above issue.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message