hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Grover <>
Subject Re: subquery + lateral view fails without count
Date Tue, 24 Apr 2012 23:39:52 GMT
Hi Ruben,
Looks like pastie is down ( because of recent DDOS attacks. Can you please
post your queries elsewhere?


Mark Grover, Business Intelligence Analyst
OANDA Corporation 

www: www: 

"Best Trading Platform" - World Finance's Forex Awards 2009. 
"The One to Watch" - Treasury Today's Adam Smith Awards 2009. 

----- Original Message -----
From: "Ruben de Vries" <>
Sent: Monday, April 23, 2012 9:08:16 AM
Subject: subquery + lateral view fails without count

It’s a bit of a weird case but I thought I might share it and hopefully find someone who
can confirm this to be a bug or tell me I should do things differently! 

Here you can find a pastie with the full create and select queries:

I’ve got two tables: 

`visit_stats` with cols date_int, memberId and parts 

`member_map` with cols member_id, gender 

I use LATERAL VIEW explode(parts) to explode the parts MAP and group on those and I want to
JOIN the member_map onto the visit_stats to group by gender. 

Since LATERAL VIEW and JOIN together aren’t supported I’m doing the join in a subquery
and then LATERAL VIEW onto that. 

It’s working as long as my query contains a COUNT ( memberId ), but if I removed that from
the result it will crash with the error below: 

java.lang.RuntimeException: Error in configuring object 

at org.apache.hadoop.util.ReflectionUtils.setJobConf( 

at org.apache.hadoop.util.ReflectionUtils.setConf( 

at org.apache.hadoop.util.ReflectionUtils.newInstance( 

at org.apache.hadoop.mapred.ReduceTask.runOldReducer( 


at org.apache.hadoop.mapred.Child$ 

at Method) 



at org.apache.hadoop.mapred.Child.main( 

Caused by: java.lang.reflect.InvocationTargetException 

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 

at sun.reflect.NativeMethodAccessorImpl.invoke( 

at sun.reflect.DelegatingMethodAccessorImpl.invoke( 

at java.lang.reflect.Method.invoke( 

at org.apache.hadoop.util.ReflectionUtils.setJobConf( 

... 9 more 

Caused by: java.lang.RuntimeException: Reduce operator initialization failed 

at org.apache.hadoop.hive.ql.exec.ExecReducer.configure( 

... 14 more 

Caused by: java.lang.RuntimeException: cannot find field _col1 from [0:_col4, 1:_col6, 2:_col10]

at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(

at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(

at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(

at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators( 

at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(

at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp( 

at org.apache.hadoop.hive.ql.exec.Operator.initialize( 

at org.apache.hadoop.hive.ql.exec.Operator.initialize( 

at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren( 

at org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp( 

at org.apache.hadoop.hive.ql.exec.Operator.initialize( 

at org.apache.hadoop.hive.ql.exec.ExecReducer.configure( 

... 14 more 

View raw message