Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 15 Dec 2015 17:09:46 +0000 (UTC)
From: "Nicholas Brenwald (JIRA)" <jira@apache.org>
To: dev@hive.apache.org
Message-ID: <JIRA.12922042.1450199341000.28436.1450199386845@Atlassian.JIRA>
In-Reply-To: <JIRA.12922042.1450199341000@Atlassian.JIRA>
References: <JIRA.12922042.1450199341000@Atlassian.JIRA>
 <JIRA.12922042.1450199341162@arcas>
Subject: [jira] [Created] (HIVE-12678) BETWEEN relational operator sometimes
 returns incorrect results against PARQUET tables
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Nicholas Brenwald created HIVE-12678:
----------------------------------------

             Summary: BETWEEN relational operator sometimes returns incorrect results against PARQUET tables
                 Key: HIVE-12678
                 URL: https://issues.apache.org/jira/browse/HIVE-12678
             Project: Hive
          Issue Type: Bug
    Affects Versions: 1.1.0
            Reporter: Nicholas Brenwald


When querying a parquet table, the BETWEEN relational operator returns incorrect results when hive.optimize.index.filter and hive.optimize.ppd.storage are enabled

Create a parquet table:
{code}
create table t(c string) stored as parquet;
{code}

Insert some strings representing dates
{code}
insert into t select '2015-12-09' from default.dual limit 1;
insert into t select '2015-12-10' from default.dual limit 1;
insert into t select '2015-12-11' from default.dual limit 1;
{code}

h3. Example 1
This query correctly returns 3:
{code}
set hive.optimize.index.filter=true;
set hive.optimize.ppd.storage=true;
select count(*) from t where c >= '2015-12-09' and c <= '2015-12-11';
+------+--+
| _c0  |
+------+--+
| 3    |
+------+--+
{code}

This query incorrectly returns 1:
{code}
set hive.optimize.index.filter=true;
set hive.optimize.ppd.storage=true;
select count(*) from t where c between '2015-12-09' and '2015-12-11';
+------+--+
| _c0  |
+------+--+
| 1    |
+------+--+
{code}

Disabling hive.optimize.findex.filter resolves the problem. This query now correctly returns 3:
{code}
set hive.optimize.index.filter=false;
set hive.optimize.ppd.storage=true;
select count(*) from t where c between '2015-12-09' and '2015-12-11';
+------+--+
| _c0  |
+------+--+
| 3    |
+------+--+
{code}

Disabling hive.optimize.ppd.storage resolves the problem. This query now correctly returns 3:
{code}
set hive.optimize.index.filter=true;
set hive.optimize.ppd.storage=false;
select count(*) from t where c between '2015-12-09' and '2015-12-11';
+------+--+
| _c0  |
+------+--+
| 3    |
+------+--+
{code}

h3. Example 2
This query correctly returns 1:
{code}
set hive.optimize.index.filter=true;
set hive.optimize.ppd.storage=true;
select count(*) from t where c >=  '2015-12-10' and c <= '2015-12-10';
+------+--+
| _c0  |
+------+--+
| 1    |
+------+--+
{code}

This query incorrectly returns 0:
{code}
set hive.optimize.index.filter=true;
set hive.optimize.ppd.storage=true;
select count(*) from t where c between '2015-12-10' and '2015-12-10';
+------+--+
| _c0  |
+------+--+
| 0    |
+------+--+
{code}

Disabling hive.optimize.findex.filter resolves the problem. This query now correctly returns 1:
{code}
set hive.optimize.index.filter=false;
set hive.optimize.ppd.storage=true;
select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
+------+--+
| _c0  |
+------+--+
| 1    |
+------+--+
{code}

Disabling hive.optimize.ppd.storage resolves the problem. This query now correctly returns 1:
{code}
set hive.optimize.index.filter=true;
set hive.optimize.ppd.storage=false;
select count(*) from t where c >= '2015-12-10' and c <= '2015-12-10';
+------+--+
| _c0  |
+------+--+
| 1    |
+------+--+
{code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)