hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2820) Invalid tag is used for MapJoinProcessor
Date Sat, 19 Jan 2013 01:21:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13557833#comment-13557833
] 

Hudson commented on HIVE-2820:
------------------------------

Integrated in Hive-trunk-hadoop2 #73 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/73/])
    HIVE-3909 Wrong data due to HIVE-2820
(Navis via namit) (Revision 1435281)

     Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1435281
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/CommonJoinOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ConditionalTask.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/LocalMapJoinProcFactory.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java
* /hive/trunk/ql/src/test/queries/clientpositive/join_reorder4.q
* /hive/trunk/ql/src/test/results/clientpositive/join_reorder4.q.out

                
> Invalid tag is used for MapJoinProcessor
> ----------------------------------------
>
>                 Key: HIVE-2820
>                 URL: https://issues.apache.org/jira/browse/HIVE-2820
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.9.0, 0.10.0
>         Environment: ubuntu
>            Reporter: Navis
>            Assignee: Navis
>             Fix For: 0.11.0
>
>         Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2820.D1935.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2820.D1935.2.patch,
ASF.LICENSE.NOT.GRANTED--HIVE-2820.D1935.3.patch, HIVE-2820.D1935.4.patch
>
>
> Testing HIVE-2810, I've found tag and alias are used in very confusing manner. For example,
query below fails..
> {code}
> hive> set hive.auto.convert.join=true;                                           
                                         
> hive> select /*+ STREAMTABLE(a) */ * from myinput1 a join myinput1 b on a.key=b.key
join myinput1 c on a.key=c.key;        
> Total MapReduce jobs = 4
> Ended Job = 1667415037, job is filtered out (removed at runtime).
> Ended Job = 1739566906, job is filtered out (removed at runtime).
> Ended Job = 1113337780, job is filtered out (removed at runtime).
> 12/02/24 10:27:14 WARN conf.HiveConf: DEPRECATED: Ignoring hive-default.xml found on
the CLASSPATH at /home/navis/hive/conf/hive-default.xml
> Execution log at: /tmp/navis/navis_20120224102727_cafe0d8d-9b21-441d-bd4e-b83303b31cdc.log
> 2012-02-24 10:27:14	Starting to launch local task to process map join;	maximum memory
= 932118528
> java.lang.NullPointerException
> 	at org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.processOp(HashTableSinkOperator.java:312)
> 	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
> 	at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> 	at org.apache.hadoop.hive.ql.exec.MapredLocalTask.startForward(MapredLocalTask.java:325)
> 	at org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:272)
> 	at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:685)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Execution failed with exit status: 2
> Obtaining error information
> {code}
> Failed task has a plan which doesn't make sense.
> {noformat}
>   Stage: Stage-8
>     Map Reduce Local Work
>       Alias -> Map Local Tables:
>         b 
>           Fetch Operator
>             limit: -1
>         c 
>           Fetch Operator
>             limit: -1
>       Alias -> Map Local Operator Tree:
>         b 
>           TableScan
>             alias: b
>             HashTable Sink Operator
>               condition expressions:
>                 0 {key} {value}
>                 1 {key} {value}
>                 2 {key} {value}
>               handleSkewJoin: false
>               keys:
>                 0 [Column[key]]
>                 1 [Column[key]]
>                 2 [Column[key]]
>               Position of Big Table: 0
>         c 
>           TableScan
>             alias: c
>             Map Join Operator
>               condition map:
>                    Inner Join 0 to 1
>                    Inner Join 0 to 2
>               condition expressions:
>                 0 {key} {value}
>                 1 {key} {value}
>                 2 {key} {value}
>               handleSkewJoin: false
>               keys:
>                 0 [Column[key]]
>                 1 [Column[key]]
>                 2 [Column[key]]
>               outputColumnNames: _col0, _col1, _col4, _col5, _col8, _col9
>               Position of Big Table: 0
>               Select Operator
>                 expressions:
>                       expr: _col0
>                       type: int
>                       expr: _col1
>                       type: int
>                       expr: _col4
>                       type: int
>                       expr: _col5
>                       type: int
>                       expr: _col8
>                       type: int
>                       expr: _col9
>                       type: int
>                 outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
>                 File Output Operator
>                   compressed: false
>                   GlobalTableId: 0
>                   table:
>                       input format: org.apache.hadoop.mapred.TextInputFormat
>                       output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
>   Stage: Stage-4
>     Map Reduce
>       Alias -> Map Operator Tree:
>         a 
>           TableScan
>             alias: a
>             HashTable Sink Operator
>               condition expressions:
>                 0 {key} {value}
>                 1 {key} {value}
>                 2 {key} {value}
>               handleSkewJoin: false
>               keys:
>                 0 [Column[key]]
>                 1 [Column[key]]
>                 2 [Column[key]]
>               Position of Big Table: 0
>       Local Work:
>         Map Reduce Local Work
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message