Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 22 Dec 2016 00:09:58 +0000 (UTC)
From: "Chao Sun (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13029553.1482265811000.576694.1482365398447@Atlassian.JIRA>
In-Reply-To: <JIRA.13029553.1482265811000@Atlassian.JIRA>
References: <JIRA.13029553.1482265811000@Atlassian.JIRA> <JIRA.13029553.1482265811608@arcas>
Subject: [jira] [Updated] (HIVE-15477) Provide options to adjust filter
 stats when column stats are not available
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 22 Dec 2016 00:10:04 -0000


     [ https://issues.apache.org/jira/browse/HIVE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chao Sun updated HIVE-15477:
----------------------------
    Attachment: HIVE-15477.2.patch

> Provide options to adjust filter stats when column stats are not available
> --------------------------------------------------------------------------
>
>                 Key: HIVE-15477
>                 URL: https://issues.apache.org/jira/browse/HIVE-15477
>             Project: Hive
>          Issue Type: Bug
>          Components: Statistics
>    Affects Versions: 2.2.0
>            Reporter: Chao Sun
>            Assignee: Chao Sun
>         Attachments: HIVE-15477.1.patch, HIVE-15477.2.patch
>
>
> Currently when column stats are not available, Hive will assume the "worst" case by setting the # of output rows to be 1/2 of the # of input rows, for each predicate expression. This could be inaccurate, especially in the presence of multiple predicates chained by AND. We have found in some cases this could cause map join to have wrong ordering and thus fail with memory issue.
> One suggestion is to provide a config (such as {{hive.stats.filter.factor}}) that can be used to control the percentage of rows emitted by a predicate expression. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)