hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11768) java.io.DeleteOnExitHook leaks memory on long running Hive Server2 Instances
Date Tue, 13 Oct 2015 21:42:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955748#comment-14955748
] 

Thejas M Nair commented on HIVE-11768:
--------------------------------------

I agree HIVE-6091 will not fix the memory leak. Also, looks like that patch missed the deletion
of tmpErrOutputFile.

Thanks for the patch [~navis]! I have added some minor comments in RB .

I also see that there is a call in Context.java and SessionState to FileSystem.deleteOnExit
 and we don't call cancelDeleteOnExit on those. Looks like those Path objects can also consume
excess memory in a long running HS2. It does not have to be addressed in this jira, we can
create a follow up jira for that.


> java.io.DeleteOnExitHook leaks memory on long running Hive Server2 Instances
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-11768
>                 URL: https://issues.apache.org/jira/browse/HIVE-11768
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 1.2.1
>            Reporter: Nemon Lou
>            Assignee: Navis
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HIVE-11768.1.patch.txt, HIVE-11768.2.patch.txt, HIVE-11768.3.patch.txt,
HIVE-11768.4.patch.txt
>
>
>   More than 490,000 paths was added to java.io.DeleteOnExitHook on one of our long running
HiveServer2 instances,taken up more than 100MB on heap.
>   Most of the paths contains a suffix of ".pipeout".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message