drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4065) ImpersonationUtil always creates new UserGroupInformation (thus new FileSystem objects), causing excessive number of threads
Date Wed, 11 Nov 2015 01:02:10 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999716#comment-14999716
] 

Jacques Nadeau commented on DRILL-4065:
---------------------------------------

I'm not an expert here but it looks like the FileSystem cache keys off of UGI.subject + URI
+ conf. The cache uses identity comparison for subject. Since we don't close FileSystem objects
(and each FileSystem object gets at least one thread), this could be a major problem for production
customers, not just the test cases. I'm not sure if the cache behavior changed for Hadoop
2.7 vs 2.4 that we upgraded in this release. Others that know the FileSystem code better might
have additional insights.

> ImpersonationUtil always creates new UserGroupInformation (thus new FileSystem objects),
causing excessive number of threads
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4065
>                 URL: https://issues.apache.org/jira/browse/DRILL-4065
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Jacques Nadeau
>            Assignee: Venki Korukanti
>             Fix For: 1.3.0
>
>         Attachments: DRILL-4065.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message