hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HIVE-17586) Make HS2 BackgroundOperationPool not fixed
Date Fri, 22 Sep 2017 21:29:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-17586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xuefu Zhang reassigned HIVE-17586:
----------------------------------


> Make HS2 BackgroundOperationPool not fixed
> ------------------------------------------
>
>                 Key: HIVE-17586
>                 URL: https://issues.apache.org/jira/browse/HIVE-17586
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 1.1.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>
> Currently the threadpool for background asynchronous operatons has a fixed size controled
by {{hive.server2.async.exec.threads}}. However, the thread factory supplied for this threadpool
is {{ThreadFactoryWithGarbageCleanup}} which creates ThreadWithGarbageCleanup. Since this
is a fixed threadpool, the thread is actually never killed, defecting the purpose of garbage
cleanup as noted in the thread class name. On the other hand, since these threads never go
away, significant resources such as threadlocal variables (classloaders, hiveconfs, etc) are
holding up even if there is no operation running. This can lead to escalated HS2 memory usage.
> Ideally, the threadpool should not be fixed, allowing thread to die out so resources
can be reclaimed. The existing config {{hive.server2.async.exec.threads}} is treated as the
max, and we can add a min for the threadpool {{hive.server2.async.exec.min.threads}}. Default
value for this configure is -1, which keeps the existing behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message