pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-5387) Test failures on JRE 11
Date Fri, 10 May 2019 22:01:00 GMT

    [ https://issues.apache.org/jira/browse/PIG-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837645#comment-16837645
] 

Rohini Palaniswamy commented on PIG-5387:
-----------------------------------------

If registerNewResource is same code, can it be moved to test Utils class. 

bq. But I don't know much about these class loader handling
  Not familiar with what changed in jdk 11 regarding classloading either. Nandor can you explain
what is the reason behind those changes and leave few lines of comments in the code itself.
Does the behavior change cause any issues with register command in pig adding files to classloader
?

> Test failures on JRE 11
> -----------------------
>
>                 Key: PIG-5387
>                 URL: https://issues.apache.org/jira/browse/PIG-5387
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.17.0
>            Reporter: Nandor Kollar
>            Priority: Major
>         Attachments: PIG-5387_1.patch
>
>
> I tried to compile Pig with JDK 8 and execute the test with Java 11, and faced with several
test failures. For example TestCommit#testCheckin2 failed with the following exception:
> {code}
> 2019-05-08 16:06:09,712 WARN  [Thread-108] mapred.LocalJobRunner (LocalJobRunner.java:run(590))
- job_local1000317333_0003
> java.lang.Exception: java.io.IOException: Deserialization error: null
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552)
> Caused by: java.io.IOException: Deserialization error: null
> 	at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:62)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.setup(PigGenericMapBase.java:183)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> 	at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.NullPointerException
> 	at org.apache.pig.impl.plan.Operator.hashCode(Operator.java:106)
> 	at java.base/java.util.HashMap.hash(HashMap.java:339)
> 	at java.base/java.util.HashMap.readObject(HashMap.java:1461)
> 	at java.base/jdk.internal.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 	at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1160)
> {code}
> It deserialization of one of the map plan failed, it appears we ran into [JDK-8201131|https://bugs.openjdk.java.net/browse/JDK-8201131].
I seems that the workaround in the issue report works, adding a readObject method to org.apache.pig.impl.plan.Operator:
> {code}
>     private void readObject(ObjectInputStream in) throws ClassNotFoundException, IOException
{
>         in.defaultReadObject();
>     }
> {code}
> solves the problem, however I'm not sure that this is the optimal solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message