hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saket Saurabh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-14448) Queries with predicate fail when ETL split strategy is chosen for ACID tables
Date Fri, 05 Aug 2016 23:50:20 GMT
Saket Saurabh created HIVE-14448:
------------------------------------

             Summary: Queries with predicate fail when ETL split strategy is chosen for ACID
tables
                 Key: HIVE-14448
                 URL: https://issues.apache.org/jira/browse/HIVE-14448
             Project: Hive
          Issue Type: Bug
          Components: Transactions
    Affects Versions: 2.2.0
            Reporter: Saket Saurabh


When ETL split strategy is applied to ACID tables with predicate pushdown (SARG enabled),
split generation fails for ACID. This bug will be usually exposed when working with data at
scale, because in most otherwise cases only BI split strategy is chosen. My guess is that
this is happening because the correct readerSchema is not being picked up when we try to extract
SARG column names.

Quickest way to reproduce is to add the following unit test to ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java

{code:title=ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java|borderStyle=solid}
 @Test
  public void testETLSplitStrategyForACID() throws Exception {
    hiveConf.setVar(HiveConf.ConfVars.HIVE_ORC_SPLIT_STRATEGY, "ETL");
    hiveConf.setBoolVar(HiveConf.ConfVars.HIVEOPTINDEXFILTER, true);
    runStatementOnDriver("insert into " + Table.ACIDTBL + " values(1,2)");
    runStatementOnDriver("alter table " + Table.ACIDTBL + " compact 'MAJOR'");
    runWorker(hiveConf);
    List<String> rs = runStatementOnDriver("select * from " +  Table.ACIDTBL  +
" where a = 1");
    int[][] resultData = new int[][] {{1,2}};
    Assert.assertEquals(stringifyValues(resultData), rs);
  }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message