Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 3 Feb 2016 05:07:39 +0000 (UTC)
From: "Vishesh Garg (JIRA)" <jira@apache.org>
To: dev@hive.apache.org
Message-ID: <JIRA.12936246.1454476019000.285478.1454476059779@Atlassian.JIRA>
In-Reply-To: <JIRA.12936246.1454476019000@Atlassian.JIRA>
References: <JIRA.12936246.1454476019000@Atlassian.JIRA>
 <JIRA.12936246.1454476019891@arcas>
Subject: [jira] [Created] (HIVE-12989) Hive with ORC - Data read from disk
 for 2 queries same despite very different selectivities
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Vishesh Garg created HIVE-12989:
-----------------------------------

             Summary: Hive with ORC - Data read from disk for 2 queries same despite very different selectivities
                 Key: HIVE-12989
                 URL: https://issues.apache.org/jira/browse/HIVE-12989
             Project: Hive
          Issue Type: Bug
          Components: ORC
    Affects Versions: 1.2.1
         Environment: TPCDS 100 GB
            Reporter: Vishesh Garg
            Priority: Minor


I'm using Hive 1.2.1 with ORC file format and doing some profiling for the data read from the disk (after clearing file system caches) for 2 queries, the data being loaded from the TPCDS benchmark. The queries are:

{code:sql}
 Q1) select count(ss_item_sk) from store_sales where ss_sold_date_sk between 2450816 and 2450817;
 Q2) select  count(ss_item_sk) from store_sales where ss_sold_date_sk between 2450816 and 2452000;
{code}

I'm using flags *hive.vectorized.execution.enabled* = true and *hive.optimize.index.filter* = true.

The output counts are about .1M and about 117M for Q1 and Q2 resp. On looking at the hive logs, I see that about 11000 row-groups are included for Q1 and 33000 row-groups are included for Q2 after filtering using index. Hence, I had expected vastly different sizes of data read. However, I see about 1GB of data read for both the queries. I suspect this has something to do with the function _addRgFilteredStreamToRanges()_


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)