Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Fri, 4 Sep 2015 00:56:45 +0000 (UTC)
From: "Thejas M Nair (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.12850160.1438231403000.253460.1441328205707@Atlassian.JIRA>
In-Reply-To: <JIRA.12850160.1438231403000@Atlassian.JIRA>
References: <JIRA.12850160.1438231403000@Atlassian.JIRA>
 <JIRA.12850160.1438231403820@arcas>
Subject: [jira] [Commented] (HIVE-11408) HiveServer2 is leaking ClassLoaders
 when add jar / temporary functions are used
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-11408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730119#comment-14730119 ] 

Thejas M Nair commented on HIVE-11408:
--------------------------------------

[~sershe] HIVE-10329 changes are large and hasn't been in use for that long. This smaller change seems safer for a backport.


> HiveServer2 is leaking ClassLoaders when add jar / temporary functions are used
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-11408
>                 URL: https://issues.apache.org/jira/browse/HIVE-11408
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.1.1
>            Reporter: Vaibhav Gumashta
>            Assignee: Vaibhav Gumashta
>         Attachments: HIVE-11408.1.patch
>
>
> I'm able to reproduce with 0.14. I'm yet to see if HIVE-10453 fixes the issue (since it's on top of a larger patch: HIVE-2573 that was added in 1.2). 
> Basically, add jar creates a new classloader for loading the classes from the new jar and adds the new classloader to the SessionState object of user's session, making the older one its parent. Creating a temporary function uses the new classloader to load the class used for the function. On closing a session, although there is code to close the classloader for the session, I'm not seeing the new classloader getting GCed and from the heapdump I can see it holds on to the temporary function's class that should have gone away after the session close. 
> Steps to reproduce:
> 1.
> {code}
> jdbc:hive2://localhost:10000/> add jar hdfs:///tmp/audf.jar;
> {code}
> 2. 
> Use a profiler (I'm using yourkit) to verify that a new URLClassLoader was added.
> 3. 
> {code}
> jdbc:hive2://localhost:10000/> CREATE TEMPORARY FUNCTION funcA AS 'org.gumashta.udf.AUDF'; 
> {code}
> 4. 
> Close the jdbc session.
> 5. 
> Take the memory snapshot and verify that the new URLClassLoader is indeed there and is holding onto the class it loaded (org.gumashta.udf.AUDF) for the session which we already closed.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)