hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11431) Vectorization: select * Left Semi Join projections NPE
Date Tue, 04 Aug 2015 06:32:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653142#comment-14653142
] 

Matt McCline commented on HIVE-11431:
-------------------------------------

Offhand, as Gopal suspected, it seems like the Join is marked as LEFT_SEMI_JOIN, but the the
smallTableIndices [1, -1, 0]  and smallTableRetainList [0, 1, 2] are asking for non-key columns.
 The Native Vector MapJoin classes VectorMapJoinLeftSemi{Long|MultiKey|String}Operator are
optimized to not output non-key columns.  Thus, the NPE.

So, looks like planner is mis-classifying the query and the Vectorizer class checking for
specialization needs to be paranoid and verify LEFT SEMI characteristics.

> Vectorization: select * Left Semi Join projections NPE
> ------------------------------------------------------
>
>                 Key: HIVE-11431
>                 URL: https://issues.apache.org/jira/browse/HIVE-11431
>             Project: Hive
>          Issue Type: Bug
>          Components: Vectorization
>    Affects Versions: 1.3.0, 1.2.1
>            Reporter: Gopal V
>            Assignee: Matt McCline
>         Attachments: left-semi-bug.sql
>
>
> The "select *" is meant to only apply to the left most table, not the right most - the
unprojected "d" from tmp1 triggers this NPE.
> {code}
> select * from tmp2 left semi join tmp1 where c1 = id and c0 = q;
> {code}
> {code}
> Caused by: java.lang.NullPointerException
>         at java.lang.System.arraycopy(Native Method)
>         at org.apache.hadoop.io.Text.set(Text.java:225)
>         at org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow$StringExtractorByValue.extract(VectorExtractRow.java:472)
>         at org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRow(VectorExtractRow.java:732)
>         at org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.process(VectorFileSinkOperator.java:96)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
>         at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:136)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
>         at org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:117)
>         at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:837)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message