hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Navis Ryu" <navis....@nexr.com>
Subject Review Request 16644: MapredLocalTask task does not make virtual columns
Date Mon, 06 Jan 2014 04:40:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16644/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-4790
    https://issues.apache.org/jira/browse/HIVE-4790


Repository: hive-git


Description
-------

>From mailing list, http://www.mail-archive.com/user@hive.apache.org/msg08264.html


{noformat}
                SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber = a.number;
fails with this error:
 
    > SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber = a.number;
Automatically selecting local only mode for query
Total MapReduce jobs = 1
setting HADOOP_USER_NAME        pmarron
13/06/25 10:52:56 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local
no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you
are connecting to a remote metastore.
Execution log at: /tmp/pmarron/.log
2013-06-25 10:52:56     Starting to launch local task to process map join;      maximum memory
= 932118528
java.lang.RuntimeException: cannot find field block__offset__inside__file from [0:rownumber,
1:offset]
        at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
        at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
        at org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldRef(DelegatedStructObjectInspector.java:74)
        at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
        at org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:68)
        at org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.initializeOp(HashTableSinkOperator.java:222)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
        at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
        at org.apache.hadoop.hive.ql.exec.MapredLocalTask.initializeOperators(MapredLocalTask.java:394)
        at org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:277)
        at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:676)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 2
{noformat}


Diffs
-----

  ql/pom.xml 90dfb2c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 99b062f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java ffc4c42 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java 770d904 
  ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java aa8f19c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/PartitionKeySampler.java 166461a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 81a1232 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecMapperContext.java 74bc2d2 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 0cc90d0 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsWork.java 3cae727 
  ql/src/test/queries/clientpositive/join_vc.q 63b3da7 
  ql/src/test/results/clientpositive/join_vc.q.out cdbc5c1 

Diff: https://reviews.apache.org/r/16644/diff/


Testing
-------


Thanks,

Navis Ryu


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message