pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Szita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-5387) Test failures on JRE 11
Date Mon, 20 May 2019 07:52:00 GMT

    [ https://issues.apache.org/jira/browse/PIG-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843746#comment-16843746
] 

Adam Szita commented on PIG-5387:
---------------------------------

Thanks Nandor, +1 for [^PIG-5387_3.patch]. [~rohini], [~knoguchi] any objections?

> Test failures on JRE 11
> -----------------------
>
>                 Key: PIG-5387
>                 URL: https://issues.apache.org/jira/browse/PIG-5387
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.17.0
>            Reporter: Nandor Kollar
>            Assignee: Nandor Kollar
>            Priority: Major
>         Attachments: PIG-5387_1.patch, PIG-5387_2.patch, PIG-5387_3.patch
>
>
> I tried to compile Pig with JDK 8 and execute the test with Java 11, and faced with several
test failures. For example TestCommit#testCheckin2 failed with the following exception:
> {code}
> 2019-05-08 16:06:09,712 WARN  [Thread-108] mapred.LocalJobRunner (LocalJobRunner.java:run(590))
- job_local1000317333_0003
> java.lang.Exception: java.io.IOException: Deserialization error: null
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552)
> Caused by: java.io.IOException: Deserialization error: null
> 	at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:62)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.setup(PigGenericMapBase.java:183)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> 	at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.NullPointerException
> 	at org.apache.pig.impl.plan.Operator.hashCode(Operator.java:106)
> 	at java.base/java.util.HashMap.hash(HashMap.java:339)
> 	at java.base/java.util.HashMap.readObject(HashMap.java:1461)
> 	at java.base/jdk.internal.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 	at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1160)
> {code}
> It deserialization of one of the map plan failed, it appears we ran into [JDK-8201131|https://bugs.openjdk.java.net/browse/JDK-8201131].
I seems that the workaround in the issue report works, adding a readObject method to org.apache.pig.impl.plan.Operator:
> {code}
>     private void readObject(ObjectInputStream in) throws ClassNotFoundException, IOException
{
>         in.defaultReadObject();
>     }
> {code}
> solves the problem, however I'm not sure that this is the optimal solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message