hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HADOOP-12707) key of FileSystem inner class Cache contains UGI.hascode which uses the defualt hascode method, leading to the memory leak
Date Thu, 14 Jan 2016 06:28:39 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth resolved HADOOP-12707.
------------------------------------
    Resolution: Won't Fix

Hello, [~haitao-tony].  Thank you for filing the issue, but the current implementation of
{{UserGroupInformation#hashCode}} and {{UserGroupInformation#equals}} is by design.  Please
see issue HADOOP-6670 if you're interested in more background.

I have 2 suggestions to offer for the memory leak problem that you noticed:
# Consider using the {{FileSystem#closeAllForUGI}} method, which would close all cached instances
for a user.
# If it's not possible to change code, then consider disabling the cache by setting configuration
property {{fs.hdfs.impl.disable.cache}} to {{true}} in core-site.xml.  This would bypass the
cache completely, which might be appropriate depending on your application.

> key of FileSystem inner class Cache contains UGI.hascode which uses the defualt hascode
method, leading to the memory leak
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-12707
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12707
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.7.1
>            Reporter: sunhaitao
>            Assignee: sunhaitao
>
> FileSystem.get(conf) method,By default it will get the fs object from CACHE,But the key
of the CACHE  constains ugi.hashCode, which uses the default hascode method of subject instead
of the hascode method overwritten by subject.
>    @Override
>       public int hashCode() {
>         return (scheme + authority).hashCode() + ugi.hashCode() + (int)unique;
>       }
> In this case, even if same user, if the calll FileSystem.get(conf) twice, two different
key will be created. In long duartion, this will lead to memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message