hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17304) ThreadMXBean based memory allocation monitory for hash table loader
Date Tue, 29 Aug 2017 20:59:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146111#comment-16146111
] 

Sergey Shelukhin commented on HIVE-17304:
-----------------------------------------

+1 with some testing

> ThreadMXBean based memory allocation monitory for hash table loader
> -------------------------------------------------------------------
>
>                 Key: HIVE-17304
>                 URL: https://issues.apache.org/jira/browse/HIVE-17304
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-17304.1.patch
>
>
> Hash table memory monitoring is based on java data model which can be unreliable because
of various reasons (wrong object size estimation, adding new variables to any class without
accounting its size for memory monitoring, etc.). We can use allocation size per thread that
is provided by ThreadMXBean and fallback to DataModel in case if JDK doesn't support thread
based allocations. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message