hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-20321) Vectorization: Cut down memory size of 1 col VectorHashKeyWrapper to <1 CacheLine
Date Wed, 15 Aug 2018 20:42:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matt McCline updated HIVE-20321:
--------------------------------
    Status: Patch Available  (was: In Progress)

> Vectorization: Cut down memory size of 1 col VectorHashKeyWrapper to <1 CacheLine
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-20321
>                 URL: https://issues.apache.org/jira/browse/HIVE-20321
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gopal V
>            Assignee: Matt McCline
>            Priority: Major
>         Attachments: HIVE-20321.01.patch, HIVE-20321.02.patch
>
>
> With a full sized LLAP instance, the memory size of the VectorHashKeyWrapper is bigger
than the low Xmx JVMs.
> {code}
> ***** 64-bit VM: **********************************************************
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper object internals:
>  OFFSET  SIZE                                                                     TYPE
DESCRIPTION                                  VALUE
>       0    16                                                                       
  (object header)                              N/A
>      16     4                                                                      int
VectorHashKeyWrapper.hashcode                N/A
>      20     4                                                                       
  (alignment/padding gap)                     
>      24     8                                                                   long[]
VectorHashKeyWrapper.longValues              N/A
>      32     8                                                                 double[]
VectorHashKeyWrapper.doubleValues            N/A
>      40     8                                                                 byte[][]
VectorHashKeyWrapper.byteValues              N/A
>      48     8                                                                    int[]
VectorHashKeyWrapper.byteStarts              N/A
>      56     8                                                                    int[]
VectorHashKeyWrapper.byteLengths             N/A
>      64     8                   org.apache.hadoop.hive.serde2.io.HiveDecimalWritable[]
VectorHashKeyWrapper.decimalValues           N/A
>      72     8                                                     java.sql.Timestamp[]
VectorHashKeyWrapper.timestampValues         N/A
>      80     8                 org.apache.hadoop.hive.common.type.HiveIntervalDayTime[]
VectorHashKeyWrapper.intervalDayTimeValues   N/A
>      88     8                                                                boolean[]
VectorHashKeyWrapper.isNull                  N/A
>      96     8   org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper.HashContext
VectorHashKeyWrapper.hashCtx                 N/A
> Instance size: 104 bytes
> Space losses: 4 bytes internal + 0 bytes external = 4 bytes total
> {code}
> Pulling this up to a parent class allows for this to be cut down to 32 bytes for the
single column case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message