hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "binlijin (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-2497) partition pruning prune some right partition under specific conditions
Date Wed, 12 Oct 2011 02:33:12 GMT

     [ https://issues.apache.org/jira/browse/HIVE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

binlijin updated HIVE-2497:
---------------------------

    Labels: patch  (was: )
    Status: Patch Available  (was: Open)
    
> partition pruning  prune some right partition under specific conditions
> -----------------------------------------------------------------------
>
>                 Key: HIVE-2497
>                 URL: https://issues.apache.org/jira/browse/HIVE-2497
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.7.1
>            Reporter: binlijin
>              Labels: patch
>             Fix For: 0.9.0
>
>
> create table src3(key string, value string) partitioned by (pt string)
> row format delimited fields terminated by ',';
> ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110911000000') ;
> ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110912000000') ;
> ALTER TABLE src3 ADD IF NOT EXISTS PARTITION (pt='20110913000000') ;
> explain extended
> select user_id 
> from
>  (
>    select 
>     cast(key as int) as user_id
>     ,case when (value like 'aaa%' or value like 'vvv%')
>             then 1
>             else 0  end as tag_student
>    from src3
>  ) sub
> where sub.tag_student > 0;
> STAGE DEPENDENCIES:
>   Stage-1 is a root stage
>   Stage-0 is a root stage
> STAGE PLANS:
>   Stage: Stage-1
>     Map Reduce
>       Alias -> Map Operator Tree:
>         sub:src3 
>           TableScan
>             alias: src3
>             Filter Operator
>               isSamplingPred: false
>               predicate:
>                   expr: (CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN
(1) ELSE (0) END > 0)
>                   type: boolean
>               Select Operator
>                 expressions:
>                       expr: UDFToInteger(key)
>                       type: int
>                       expr: CASE WHEN (((value like 'aaa%') or (value like 'vvv%')))
THEN (1) ELSE (0) END
>                       type: int
>                 outputColumnNames: _col0, _col1
>                 Filter Operator
>                   isSamplingPred: false
>                   predicate:
>                       expr: (_col1 > 0)
>                       type: boolean
>                   Select Operator
>                     expressions:
>                           expr: _col0
>                           type: int
>                     outputColumnNames: _col0
>                     File Output Operator
>                       compressed: false
>                       GlobalTableId: 0
>                       directory: hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-26-12_894_9085644225727185586/-ext-10001
>                       NumFilesPerFileSink: 1
>                       table:
>                           input format: org.apache.hadoop.mapred.TextInputFormat
>                           output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
>                           properties:
>                             columns _col0
>                             columns.types int
>                             serialization.format 1
>                       TotalFiles: 1
>                       MultiFileSpray: false
>       Needs Tagging: false
>   Stage: Stage-0
>     Fetch Operator
>       limit: -1
> if we set hive.optimize.ppd=false;
> STAGE DEPENDENCIES:
>   Stage-1 is a root stage
>   Stage-0 is a root stage
> STAGE PLANS:
>   Stage: Stage-1
>     Map Reduce
>       Alias -> Map Operator Tree:
>         sub:src3 
>           TableScan
>             alias: src3
>             Select Operator
>               expressions:
>                     expr: UDFToInteger(key)
>                     type: int
>                     expr: CASE WHEN (((value like 'aaa%') or (value like 'vvv%'))) THEN
(1) ELSE (0) END
>                     type: int
>               outputColumnNames: _col0, _col1
>               Filter Operator
>                 isSamplingPred: false
>                 predicate:
>                     expr: (_col1 > 0)
>                     type: boolean
>                 Select Operator
>                   expressions:
>                         expr: _col0
>                         type: int
>                   outputColumnNames: _col0
>                   File Output Operator
>                     compressed: false
>                     GlobalTableId: 0
>                     directory: hdfs://localhost:54310/tmp/hive-tianzhao/hive_2011-10-11_19-27-22_527_1729287213481398480/-ext-10001
>                     NumFilesPerFileSink: 1
>                     table:
>                         input format: org.apache.hadoop.mapred.TextInputFormat
>                         output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
>                         properties:
>                           columns _col0
>                           columns.types int
>                           serialization.format 1
>                     TotalFiles: 1
>                     MultiFileSpray: false
>       Needs Tagging: false
>       Path -> Alias:
>         hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110911000000 [sub:src3]
>         hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110912000000 [sub:src3]
>         hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110913000000 [sub:src3]
>       Path -> Partition:
>         hdfs://localhost:54310/user/hive/warehouse/src3/pt=20110911000000 
>           Partition
>             base file name: pt=20110911000000

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message