hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12707) key of FileSystem inner class Cache contains UGI.hascode which uses the default hascode method, leading to the memory leak with Proxy Users
Date Mon, 04 Jun 2018 17:22:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500549#comment-16500549
] 

Steve Loughran commented on HADOOP-12707:
-----------------------------------------

Added proxy users to the title.
 [~cnauroth]: what are your thoughts here?

[~xinlishang] I think you may want to consider what you could do with  your own cache, as
that would isolate your code & you could be clever here. That is; you can cache filesystems
outside FileSystem.cache if you really want to

> key of FileSystem inner class Cache contains UGI.hascode which uses the default hascode
method, leading to the memory leak with Proxy Users
> -------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-12707
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12707
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.7.1
>            Reporter: sunhaitao
>            Assignee: sunhaitao
>            Priority: Major
>
> FileSystem.get(conf) method,By default it will get the fs object from CACHE,But the key
of the CACHE  constains ugi.hashCode, which uses the default hascode method of subject instead
of the hascode method overwritten by subject.
>    @Override
>       public int hashCode() {
>         return (scheme + authority).hashCode() + ugi.hashCode() + (int)unique;
>       }
> In this case, even if same user, if the calll FileSystem.get(conf) twice, two different
key will be created. In long duartion, this will lead to memory leak.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message