hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <>
Subject [jira] [Created] (HIVE-13873) Column pruning for nested fields
Date Fri, 27 May 2016 04:59:12 GMT
Xuefu Zhang created HIVE-13873:

             Summary: Column pruning for nested fields
                 Key: HIVE-13873
             Project: Hive
          Issue Type: New Feature
          Components: Logical Optimizer
            Reporter: Xuefu Zhang

Some columnar file formats such as Parquet store fields in struct type also column by column
using encoding described in Google Dramel pager. It's very common in big data where data are
stored in structs while queries only needs a subset of the the fields in the structs. However,
presently Hive still needs to read the whole struct regardless whether all fields are selected.
Therefore, pruning unwanted sub-fields in struct or nested fields at file reading time would
be a big performance boost for such scenarios.

This message was sent by Atlassian JIRA

View raw message