hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8914) HDFSCleanup thread holds reference to FileSystem
Date Fri, 21 Nov 2014 19:33:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221317#comment-14221317
] 

Eugene Koifman commented on HIVE-8914:
--------------------------------------

[~shanyu], are you arguing that you don't need to call {{FileSystem.close()}} or {{closeAll()}}
?

> HDFSCleanup thread holds reference to FileSystem
> ------------------------------------------------
>
>                 Key: HIVE-8914
>                 URL: https://issues.apache.org/jira/browse/HIVE-8914
>             Project: Hive
>          Issue Type: Bug
>          Components: WebHCat
>    Affects Versions: 0.13.1
>            Reporter: shanyu zhao
>            Assignee: shanyu zhao
>         Attachments: HIVE-8914.patch
>
>
> WebHCat server has a long running cleanup thread (HDFSCleanup) which holds a reference
to FileSystem. Because of this reference, many FileSystem related objects (e.g. MetricsSystemImpl)
cannot be garbage collected. Sometimes this causes OOM exception. Since the cleanup is done
every 12 hours by default, we can simply recreate a FileSystem when we need to use it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message