Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C7CC09AD1 for ; Mon, 17 Oct 2011 06:18:38 +0000 (UTC) Received: (qmail 26825 invoked by uid 500); 17 Oct 2011 06:18:38 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 26762 invoked by uid 500); 17 Oct 2011 06:18:38 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 26561 invoked by uid 500); 17 Oct 2011 06:18:37 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 26558 invoked by uid 99); 17 Oct 2011 06:18:37 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Oct 2011 06:18:37 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Oct 2011 06:18:34 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id EEBFE30BFD4 for ; Mon, 17 Oct 2011 06:18:12 +0000 (UTC) Date: Mon, 17 Oct 2011 06:18:12 +0000 (UTC) From: "binlijin (Updated) (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: <241749496.19772.1318832292979.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <1918326447.3441.1318386551751.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Updated] (HIVE-2497) partition pruning prune some right partition under specific conditions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HIVE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HIVE-2497: --------------------------- Attachment: hive-2497.2.patch > partition pruning prune some right partition under specific conditions > ----------------------------------------------------------------------- > > Key: HIVE-2497 > URL: https://issues.apache.org/jira/browse/HIVE-2497 > Project: Hive > Issue Type: Bug > Affects Versions: 0.7.1 > Reporter: binlijin > Labels: patch > Fix For: 0.9.0 > > Attachments: hive-2497.2.patch, hive-2497.patch > > > create table src3(key string, value string) partitioned by (pt string) > row format delimited fields terminated by ','; > ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110911000000') ; > ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110912000000') ; > ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110913000000') ; > explain extended > select user_id > from > ( > select > cast(key as int) as user_id > ,case when (value like 'aaa%' or value like 'vvv%') > then 1 > else 0 end as tag_student > from src3 > ) sub > where sub.tag_student > 0; > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 is a root stage > STAGE PLANS: > Stage: Stage-1 > Map Reduce > Alias -> Map Operator Tree: > sub:src3 > TableScan > alias: src3 > Filter Operator > isSamplingPred: false > predicate: > expr: (CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN (1) ELSE (0) END > 0) > type: boolean > Select Operator > expressions: > expr: UDFToInteger(key) > type: int > expr: CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN (1) ELSE (0) END > type: int > outputColumnNames: _col0, _col1 > Filter Operator > isSamplingPred: false > predicate: > expr: (_col1 > 0) > type: boolean > Select Operator > expressions: > expr: _col0 > type: int > outputColumnNames: _col0 > File Output Operator > compressed: false > GlobalTableId: 0 > directory: hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-26-12_894_9085644225727185586/-ext-10001 > NumFilesPerFileSink: 1 > table: > input format: org.apache.hadoop.mapred.TextInputFormat > output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat > properties: > columns _col0 > columns.types int > serialization.format 1 > TotalFiles: 1 > MultiFileSpray: false > Needs Tagging: false > Stage: Stage-0 > Fetch Operator > limit: -1 > if we set hive.optimize.ppd=false; > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 is a root stage > STAGE PLANS: > Stage: Stage-1 > Map Reduce > Alias -> Map Operator Tree: > sub:src3 > TableScan > alias: src3 > Select Operator > expressions: > expr: UDFToInteger(key) > type: int > expr: CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN (1) ELSE (0) END > type: int > outputColumnNames: _col0, _col1 > Filter Operator > isSamplingPred: false > predicate: > expr: (_col1 > 0) > type: boolean > Select Operator > expressions: > expr: _col0 > type: int > outputColumnNames: _col0 > File Output Operator > compressed: false > GlobalTableId: 0 > directory: hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-27-22_527_1729287213481398480/-ext-10001 > NumFilesPerFileSink: 1 > table: > input format: org.apache.hadoop.mapred.TextInputFormat > output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat > properties: > columns _col0 > columns.types int > serialization.format 1 > TotalFiles: 1 > MultiFileSpray: false > Needs Tagging: false > Path -> Alias: > hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110911000000 [sub:src3] > hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110912000000 [sub:src3] > hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110913000000 [sub:src3] > Path -> Partition: > hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110911000000 > Partition > base file name: pt=20110911000000 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira