From dev-return-60396-archive-asf-public=cust-asf.ponee.io@pig.apache.org Fri May 10 22:01:02 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 66F18180763 for ; Sat, 11 May 2019 00:01:02 +0200 (CEST) Received: (qmail 76328 invoked by uid 500); 10 May 2019 22:01:01 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 76292 invoked by uid 500); 10 May 2019 22:01:01 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 76275 invoked by uid 99); 10 May 2019 22:01:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 May 2019 22:01:01 +0000 Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 901C2E2BC6 for ; Fri, 10 May 2019 22:01:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2613925815 for ; Fri, 10 May 2019 22:01:00 +0000 (UTC) Date: Fri, 10 May 2019 22:01:00 +0000 (UTC) From: "Rohini Palaniswamy (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PIG-5387) Test failures on JRE 11 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PIG-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837645#comment-16837645 ] Rohini Palaniswamy commented on PIG-5387: ----------------------------------------- If registerNewResource is same code, can it be moved to test Utils class. bq. But I don't know much about these class loader handling Not familiar with what changed in jdk 11 regarding classloading either. Nandor can you explain what is the reason behind those changes and leave few lines of comments in the code itself. Does the behavior change cause any issues with register command in pig adding files to classloader ? > Test failures on JRE 11 > ----------------------- > > Key: PIG-5387 > URL: https://issues.apache.org/jira/browse/PIG-5387 > Project: Pig > Issue Type: Bug > Affects Versions: 0.17.0 > Reporter: Nandor Kollar > Priority: Major > Attachments: PIG-5387_1.patch > > > I tried to compile Pig with JDK 8 and execute the test with Java 11, and faced with several test failures. For example TestCommit#testCheckin2 failed with the following exception: > {code} > 2019-05-08 16:06:09,712 WARN [Thread-108] mapred.LocalJobRunner (LocalJobRunner.java:run(590)) - job_local1000317333_0003 > java.lang.Exception: java.io.IOException: Deserialization error: null > at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492) > at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552) > Caused by: java.io.IOException: Deserialization error: null > at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:62) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.setup(PigGenericMapBase.java:183) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) > at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271) > at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > Caused by: java.lang.NullPointerException > at org.apache.pig.impl.plan.Operator.hashCode(Operator.java:106) > at java.base/java.util.HashMap.hash(HashMap.java:339) > at java.base/java.util.HashMap.readObject(HashMap.java:1461) > at java.base/jdk.internal.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at java.base/java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1160) > {code} > It deserialization of one of the map plan failed, it appears we ran into [JDK-8201131|https://bugs.openjdk.java.net/browse/JDK-8201131]. I seems that the workaround in the issue report works, adding a readObject method to org.apache.pig.impl.plan.Operator: > {code} > private void readObject(ObjectInputStream in) throws ClassNotFoundException, IOException { > in.defaultReadObject(); > } > {code} > solves the problem, however I'm not sure that this is the optimal solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)