hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vihang Karajgaonkar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17874) Parquet vectorization fails on tables with complex columns when there are no projected columns
Date Mon, 23 Oct 2017 04:36:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16214620#comment-16214620
] 

Vihang Karajgaonkar commented on HIVE-17874:
--------------------------------------------

Thanks for the review [~Ferd]. I made changes as you suggested. I moved {{colsToInclude =
ColumnProjectionUtils.getReadColumnIDs(conf);}} in the {{initialize}} method because I got
rid of unnecessary field {{indexColumnsWanted}} and reused colsToInclude instead. I have moved
the {{rbCtx = Utilities.getVectorizedRowBatchCtx(conf);}} in the initialize method as well
like you suggested. Also updated the comment and removed unnecessary diff. Feel free to let
me know if you want me to publish the patch on RB as well.

> Parquet vectorization fails on tables with complex columns when there are no projected
columns
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-17874
>                 URL: https://issues.apache.org/jira/browse/HIVE-17874
>             Project: Hive
>          Issue Type: Sub-task
>    Affects Versions: 2.2.0
>            Reporter: Vihang Karajgaonkar
>            Assignee: Vihang Karajgaonkar
>         Attachments: HIVE-17874.01-branch-2.patch, HIVE-17874.01.patch, HIVE-17874.02.patch
>
>
> When a parquet table contains an unsupported type like {{Map}}, {{LIST}} or {{UNION}}
simple queries like {{select count(*) from table}} fails with {{unsupported type exception}}
even though vectorized reader doesn't really need read the complex type into batches.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message