hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrian Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6765) ASTNodeOrigin unserializable leads to fail when join with view
Date Thu, 27 Mar 2014 10:13:16 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949118#comment-13949118
] 

Adrian Wang commented on HIVE-6765:
-----------------------------------

when I run the test case in hive command line(0.12-release), the full output is as follows,
hive> SELECT s1.x1 FROM v1 s1 JOIN (SELECT MAX(a1) AS ma FROM t1) s2 ON s1.x1 = s2.ma;
java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.XMLEncoder.writeStatement(XMLEncoder.java:361)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:277)
	at org.apache.hadoop.hive.ql.exec.Utilities.serializeObject(Utilities.java:666)
	at org.apache.hadoop.hive.ql.exec.Utilities.clonePlan(Utilities.java:637)
	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.processCurrentTask(CommonJoinTaskDispatcher.java:505)
	at org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:182)
	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
	at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:79)
	at org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:90)
	at org.apache.hadoop.hive.ql.parse.MapReduceCompiler.compile(MapReduceCompiler.java:300)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8410)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:284)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:441)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:342)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:977)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.Exception: XMLEncoder: discarding statement XMLEncoder.writeObject(MapredWork);
	... 29 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:267)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:408)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:116)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:113)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeObject1(Encoder.java:231)
	at java.beans.Encoder.cloneStatement(Encoder.java:244)
	at java.beans.Encoder.writeStatement(Encoder.java:275)
	at java.beans.XMLEncoder.writeStatement(XMLEncoder.java:348)
	... 28 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:267)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:408)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:116)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.DefaultPersistenceDelegate.doProperty(DefaultPersistenceDelegate.java:229)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:264)
	... 41 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.java_util_Map_PersistenceDelegate.initialize(MetaData.java:652)
	at java.beans.PersistenceDelegate.initialize(PersistenceDelegate.java:206)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:406)
	at java.beans.PersistenceDelegate.initialize(PersistenceDelegate.java:206)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:406)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:116)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:113)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.DefaultPersistenceDelegate.doProperty(DefaultPersistenceDelegate.java:229)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:264)
	... 49 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:267)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:408)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:116)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:113)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.java_util_Map_PersistenceDelegate.initialize(MetaData.java:643)
	... 65 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:267)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:408)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:116)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:113)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.DefaultPersistenceDelegate.doProperty(DefaultPersistenceDelegate.java:229)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:264)
	... 77 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.java_util_Map_PersistenceDelegate.initialize(MetaData.java:652)
	at java.beans.PersistenceDelegate.initialize(PersistenceDelegate.java:206)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:406)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:116)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.DefaultPersistenceDelegate.doProperty(DefaultPersistenceDelegate.java:229)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:264)
	... 90 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:267)
	at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:408)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:116)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:113)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.java_util_Map_PersistenceDelegate.initialize(MetaData.java:643)
	... 99 more
Caused by: java.lang.RuntimeException: Cannot serialize object
	at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:652)
	at java.beans.Encoder.getValue(Encoder.java:108)
	at java.beans.Encoder.get(Encoder.java:225)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:110)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:113)
	at java.beans.Encoder.writeObject(Encoder.java:74)
	at java.beans.XMLEncoder.writeObject(XMLEncoder.java:274)
	at java.beans.Encoder.writeExpression(Encoder.java:304)
	at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:389)
	at java.beans.DefaultPersistenceDelegate.doProperty(DefaultPersistenceDelegate.java:229)
	at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:264)
	... 111 more
Caused by: java.lang.InstantiationException: org.apache.hadoop.hive.ql.parse.ASTNodeOrigin
	at java.lang.Class.newInstance0(Class.java:359)
	at java.lang.Class.newInstance(Class.java:327)
	at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:55)
	at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:262)
	at java.beans.Statement.invokeInternal(Statement.java:257)
	at java.beans.Statement.access$000(Statement.java:56)
	at java.beans.Statement$2.run(Statement.java:158)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.beans.Statement.invoke(Statement.java:154)
	at java.beans.Expression.getValue(Expression.java:115)
	at java.beans.Encoder.getValue(Encoder.java:105)
	... 124 more
FAILED: SemanticException Generate Map Join Task Error: Cannot serialize object


> ASTNodeOrigin unserializable leads to fail when join with view
> --------------------------------------------------------------
>
>                 Key: HIVE-6765
>                 URL: https://issues.apache.org/jira/browse/HIVE-6765
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.12.0
>            Reporter: Adrian Wang
>         Attachments: HIVE-6765.patch.1
>
>
> when a view contains a UDF, and the view comes into a JOIN operation, Hive will encounter
a bug with stack trace like
> Caused by: java.lang.InstantiationException: org.apache.hadoop.hive.ql.parse.ASTNodeOrigin
> 	at java.lang.Class.newInstance0(Class.java:359)
> 	at java.lang.Class.newInstance(Class.java:327)
> 	at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:616)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message