From dev-return-60201-archive-asf-public=cust-asf.ponee.io@pig.apache.org Tue Jan 8 23:13:05 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id BE551180652 for ; Tue, 8 Jan 2019 23:13:04 +0100 (CET) Received: (qmail 69052 invoked by uid 500); 8 Jan 2019 22:13:03 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 69041 invoked by uid 500); 8 Jan 2019 22:13:03 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 69038 invoked by uid 99); 8 Jan 2019 22:13:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Jan 2019 22:13:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 5D187C0DBE for ; Tue, 8 Jan 2019 22:13:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id dOK_rN0rHRhV for ; Tue, 8 Jan 2019 22:13:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 2469E610EE for ; Tue, 8 Jan 2019 22:13:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 648E8E25AA for ; Tue, 8 Jan 2019 22:13:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0DC2225566 for ; Tue, 8 Jan 2019 22:13:00 +0000 (UTC) Date: Tue, 8 Jan 2019 22:13:00 +0000 (UTC) From: "Koji Noguchi (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (PIG-5375) NullPointerException for multi-level self unions with Tez UnionOptimizer MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Koji Noguchi created PIG-5375: --------------------------------- Summary: NullPointerException for multi-level self unions with Tez UnionOptimizer Key: PIG-5375 URL: https://issues.apache.org/jira/browse/PIG-5375 Project: Pig Issue Type: Bug Reporter: Koji Noguchi Assignee: Koji Noguchi {code} A = load 'input.txt' as (a0:int, a1: chararray, a2:int); B = load 'input.txt' as (a0:int, a1: chararray, a2:int); C = load 'input.txt' as (a0:int, a1: chararray, a2:int); A_and_B = UNION A, B; SPLIT A_and_B INTO A_and_B2 IF a0 > 10, A_and_B3 OTHERWISE; A_and_B_and_C = UNION ONSCHEMA C, A_and_B; X = UNION ONSCHEMA A_and_B_and_C, A_and_B2, A_and_B3; X2 = GROUP X ALL ; dump X2; {code} This fails _on Tez_ with {noformat} Pig Stack Trace --------------- ERROR 1002: Unable to store alias X2 org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias X2 at org.apache.pig.PigServer.openIterator(PigServer.java:1024) at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:790) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:383) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:231) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) at org.apache.pig.Main.run(Main.java:630) at org.apache.pig.Main.main(Main.java:175) Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias X2 at org.apache.pig.PigServer.storeEx(PigServer.java:1127) at org.apache.pig.PigServer.store(PigServer.java:1086) at org.apache.pig.PigServer.openIterator(PigServer.java:999) ... 7 more Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0: java.lang.NullPointerException at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:296) at org.apache.pig.PigServer.launchPlan(PigServer.java:1479) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1464) at org.apache.pig.PigServer.storeEx(PigServer.java:1123) ... 9 more Caused by: java.lang.NullPointerException at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezPOPackageAnnotator.patchPackage(TezPOPackageAnnotator.java:97) at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezPOPackageAnnotator.handlePackage(TezPOPackageAnnotator.java:78) at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezPOPackageAnnotator.visitTezOp(TezPOPackageAnnotator.java:61) at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:265) at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:56) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71) at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52) at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:46) at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.launchPig(TezLauncher.java:197) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:290) ... 12 more {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)