sentry-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Binglin Chang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SENTRY-705) Possible memory leak of HiveAuthzBinding.authzBindingMap
Date Sat, 09 May 2015 16:55:00 GMT

    [ https://issues.apache.org/jira/browse/SENTRY-705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536760#comment-14536760
] 

Binglin Chang commented on SENTRY-705:
--------------------------------------

The attached patch remove the additional set call to prevent memory leak. Still I think using
a global HiveAuthzBinding.authzBindingMap is not good practice, what if hive add some code
path or introduces new error, and Driver.compile get called but Driver.execute never get called,
the HiveAuthzBinding object never got cleared. 


> Possible memory leak of HiveAuthzBinding.authzBindingMap
> --------------------------------------------------------
>
>                 Key: SENTRY-705
>                 URL: https://issues.apache.org/jira/browse/SENTRY-705
>             Project: Sentry
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Binglin Chang
>         Attachments: SENTRY-705.001.patch
>
>
> We are using hive0.13 with sentry, version cdh5.2.0, noticed that hiveserver uses lot
of memory and doesn't get freed.
> Memory profiling showed that there are lots(millions) of HiveConf and HiveAuthzConf instances
hold by HiveAuthzBinding.authzBindingMap
> Look at the code, related call order:
> Driver.compile
>    HiveAuthzBindingHook.preAnalyze
>    HiveAuthzBindingHook.postAnalyze
>       hiveAuthzBinding.set
> Driver.execute
>    HiveAuthzBindingPreExecHook.run
>       hiveAuthzBinding.clear
> Between Driver.compile and Driver.execute, there are too many logic(which sentry can
not control), not sure hiveAuthzBinding.clear is guaranteed to be called



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message