hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5068) Some queries fail due to XMLEncoder error on JDK7
Date Wed, 14 Aug 2013 15:23:48 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13739765#comment-13739765
] 

Brock Noland commented on HIVE-5068:
------------------------------------

I wrote a quick and simple change to use plain old java serialization and hit the error below.
My guess is we'll have to mark some more stuff transient to do this.

{noformat}
Caused by: java.io.NotSerializableException: org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPEqual
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1181)
  at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
  at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
  at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
  at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
  at java.util.ArrayList.writeObject(ArrayList.java:710)
  at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
  at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
  at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
  at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
  at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
  at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
  at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
  at java.util.ArrayList.writeObject(ArrayList.java:710)
  at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
  at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
  at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
  at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
  at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
  at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
  at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
  at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
  at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
  at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
  at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
  at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
  at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
  at java.util.HashMap.writeObject(HashMap.java:1099)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
                
> Some queries fail due to XMLEncoder error on JDK7
> -------------------------------------------------
>
>                 Key: HIVE-5068
>                 URL: https://issues.apache.org/jira/browse/HIVE-5068
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Brock Noland
>
> Looks like something snuck in that breaks the JDK 7 build:
> {noformat}
> Caused by: java.lang.Exception: XMLEncoder: discarding statement ArrayList.add(ASTNode);
>         ... 106 more
> Caused by: java.lang.RuntimeException: Cannot serialize object
>         at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:598)
>         at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:238)
>         at java.beans.DefaultPersistenceDelegate.initialize(DefaultPersistenceDelegate.java:400)
>         at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:118)
>         at java.beans.Encoder.writeObject(Encoder.java:74)
>         at java.beans.XMLEncoder.writeObject(XMLEncoder.java:327)
>         at java.beans.Encoder.writeExpression(Encoder.java:330)
>         at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:454)
>         at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:115)
>         at java.beans.Encoder.writeObject(Encoder.java:74)
>         at java.beans.XMLEncoder.writeObject(XMLEncoder.java:327)
>         at java.beans.Encoder.writeObject1(Encoder.java:258)
>         at java.beans.Encoder.cloneStatement(Encoder.java:271)
>         at java.beans.Encoder.writeStatement(Encoder.java:301)
>         at java.beans.XMLEncoder.writeStatement(XMLEncoder.java:400)
>         ... 105 more
> Caused by: java.lang.RuntimeException: Cannot serialize object
>         at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:598)
>         at java.beans.Encoder.getValue(Encoder.java:108)
>         at java.beans.Encoder.get(Encoder.java:252)
>         at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:112)
>         at java.beans.Encoder.writeObject(Encoder.java:74)
>         at java.beans.XMLEncoder.writeObject(XMLEncoder.java:327)
>         at java.beans.Encoder.writeExpression(Encoder.java:330)
>         at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:454)
>         at java.beans.PersistenceDelegate.writeObject(PersistenceDelegate.java:115)
>         at java.beans.Encoder.writeObject(Encoder.java:74)
>         at java.beans.XMLEncoder.writeObject(XMLEncoder.java:327)
>         at java.beans.Encoder.writeExpression(Encoder.java:330)
>         at java.beans.XMLEncoder.writeExpression(XMLEncoder.java:454)
>         at java.beans.DefaultPersistenceDelegate.initBean(DefaultPersistenceDelegate.java:232)
>         ... 118 more
> Caused by: java.lang.InstantiationException: org.antlr.runtime.CommonToken
>         at java.lang.Class.newInstance(Class.java:359)
>         at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
>         at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
>         at java.beans.Statement.invokeInternal(Statement.java:292)
>         at java.beans.Statement.access$000(Statement.java:58)
>         at java.beans.Statement$2.run(Statement.java:185)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.beans.Statement.invoke(Statement.java:182)
>         at java.beans.Expression.getValue(Expression.java:153)
>         at java.beans.Encoder.getValue(Encoder.java:105)
>         ... 130 more
> {noformat}
> and
> {noformat}
> java.lang.RuntimeException: Cannot serialize object
>         at org.apache.hadoop.hive.ql.exec.Utilities$1.exceptionThrown(Utilities.java:598)
>         at java.beans.XMLEncoder.writeStatement(XMLEncoder.java:426)
>         at java.beans.XMLEncoder.writeObject(XMLEncoder.java:330)
>         at org.apache.hadoop.hive.ql.exec.Utilities.serializeObject(Utilities.java:611)
>         at org.apache.hadoop.hive.ql.plan.MapredWork.toXML(MapredWork.java:88)
>         at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinTaskDispatcher.processCurrentTask(CommonJoinTaskDispatcher.java:505)
>         at org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:182)
>         at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111)
>         at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:194)
>         at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:139)
>         at org.apache.hadoop.hive.ql.optimizer.physical.CommonJoinResolver.resolve(CommonJoinResolver.java:79)
>         at org.apache.hadoop.hive.ql.optimizer.physical.PhysicalOptimizer.optimize(PhysicalOptimizer.java:90)
>         at org.apache.hadoop.hive.ql.parse.MapReduceCompiler.compile(MapReduceCompiler.java:292)
>         at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8333)
>         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:278)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:437)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:341)
>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:973)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:885)
>         at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
>         at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
>         at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
>         at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25(TestCliDriver.java:108)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at junit.framework.TestCase.runTest(TestCase.java:154)
>         at junit.framework.TestCase.runBare(TestCase.java:127)
>         at junit.framework.TestResult$1.protect(TestResult.java:106)
>         at junit.framework.TestResult.runProtected(TestResult.java:124)
>         at junit.framework.TestResult.run(TestResult.java:109)
>         at junit.framework.TestCase.run(TestCase.java:118)
>         at junit.framework.TestSuite.runTest(TestSuite.java:208)
>         at junit.framework.TestSuite.run(TestSuite.java:203)
>         at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520)
>         at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
>         at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
> Caused by: java.lang.Exception: XMLEncoder: discarding statement XMLEncoder.writeObject(MapredWork);
>         ... 40 more
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message