hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <>
Subject [jira] [Created] (HIVE-11726) Pushed IN predicates created by PointLookupOptimizer to the metastore
Date Thu, 03 Sep 2015 16:50:47 GMT
Jesus Camacho Rodriguez created HIVE-11726:

             Summary: Pushed IN predicates created by PointLookupOptimizer to the metastore
                 Key: HIVE-11726
             Project: Hive
          Issue Type: Bug
    Affects Versions: 2.0.0
            Reporter: Jesus Camacho Rodriguez
            Assignee: Jesus Camacho Rodriguez

The PointLookupOptimizer can turn off some of the optimizations due to its use of tuple IN()

HIVE-11573 introduced the extraction of sub-clauses that could be pushed down till the TableScan
operators, though they wouldn't be pushed down to the metastore.

In this issue, we tackle this problem by:
1) Grouping the columns in the sub-clauses depending on their lineage. This way PPD will be
able to push them down throw the plan without any extension. For instance, if a, b, and c
are partition columns, a and b belong to table1, and c belong to table2:
(a,b,c) IN ((1,2,3),(2,3,4)) ->
           (a,b) IN ((1,2),(2,3)) and c in (3,4) and (a,b,c) IN ((1,2,3),(2,3,4))
2) Extending the filter parser of the metastore to support IN clauses, including multiple
columns. This allows to push those additional predicates down throw directSQL to the metastore.

This message was sent by Atlassian JIRA

View raw message