hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-9937) LLAP: Vectorized Field-By-Field Serialize / Deserialize to support new Vectorized Map Join
Date Thu, 26 Mar 2015 22:44:53 GMT

    [ https://issues.apache.org/jira/browse/HIVE-9937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382857#comment-14382857
] 

Gopal V commented on HIVE-9937:
-------------------------------

[~mmccline]: Ran a few scale tests last night and there seems to be no visible issues with
the patch from last night.

General comment about asserts - the regular runtime turns off asserts, so you should be using
Preconditions.check operations particularly if it is outside the core loop (like the futures.size).

Need to re-verify the TODO in VectorAppMasterEventOperator - make sure nothing in the super.process
actually buffers the Object[] row, since now the data is modified in-place, while earlier
it was generating a new array for each row.

This has no safety switch to turn off other than "turn off vectorization", I'd like to see
if [~mmokhtar] can get a full TPC-DS run for this.

With this epic patch, the slowest part of a group-by is now the full-sort, which gives me
something else to fix :)

> LLAP: Vectorized Field-By-Field Serialize / Deserialize to support new Vectorized Map
Join
> ------------------------------------------------------------------------------------------
>
>                 Key: HIVE-9937
>                 URL: https://issues.apache.org/jira/browse/HIVE-9937
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Matt McCline
>            Assignee: Matt McCline
>         Attachments: HIVE-9937.01.patch, HIVE-9937.02.patch, HIVE-9937.03.patch, HIVE-9937.04.patch,
HIVE-9937.05.patch, HIVE-9937.06.patch, HIVE-9937.07.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message