hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tangjunjie (JIRA)" <>
Subject [jira] [Created] (HIVE-10814) hive on tez skew table plan wrong
Date Mon, 25 May 2015 01:42:17 GMT
tangjunjie created HIVE-10814:

             Summary: hive on tez   skew table plan wrong
                 Key: HIVE-10814
             Project: Hive
          Issue Type: Bug
          Components: Query Processor
    Affects Versions: 1.1.0
         Environment: hive 1.1.0 + tez 0.53 
            Reporter: tangjunjie

set hive.execution.engine=mr; 
set hive.mapred.supports.subdirectories=true; 
set hive.optimize.skewjoin.compiletime = true; 

ALTER TABLE tandem.fct_traffic_navpage_path_detl SKEWED BY (ordr_code,cart_prod_id) ON (('','NULL'));

Vertex failed, vertexName=initialmap, vertexId=vertex_1419300485749_1514787_1_00, diagnostics=[Task
failed, taskId=task_1419300485749_1514787_1_00_000245, diagnostics=[TaskAttempt 0 failed,
info=[Error: Failure while running task:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row {"parnt_ordr_id":3715999535959,"parnt_ordr_code":"3715999535959","end_user_id":163846959,"comb_prod_id":7873715,"sale_amt":99.0,"actl_sale_amt":99.0,"sale_num":1,"updt_time":"2015-05-11


at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$

at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$

I think this is the error,cart_prod_id col not exist in table univ_parnt_tranx_comb_detl 
alias: o 
Statistics: Num rows: 109845709 Data size: 14499651703 Basic stats: PARTIAL Column stats:
Filter Operator 
predicate: (not ((ordr_code = '') and (cart_prod_id = null))) (type: boolean) 
Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE 
Reduce Output Operator 
key expressions: parnt_ordr_code (type: string), comb_prod_id (type: bigint) 
sort order: ++ 
Map-reduce partition columns: parnt_ordr_code (type: string), comb_prod_id (type: bigint)

Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE 
value expressions: end_user_id (type: bigint), actl_sale_amt (type: double), sale_num (type:
bigint), ds (type: string) 

This message was sent by Atlassian JIRA

View raw message