pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit
Date Thu, 26 Feb 2015 06:32:05 GMT

    [ https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337976#comment-14337976
] 

Daniel Dai commented on PIG-3114:
---------------------------------

The original fix does not apply to the new function introduced in PIG-2692. I cannot find
a easy fix for that. We do need to register the query multiple times to get alias mock working,
but that causes "Duplicated macro" error.

One simple work around is to change the test to register macro separately:
{code}
        String[] script = {
                "data =  LOAD 'input' AS (query:CHARARRAY);",
                "queries_group = GROUP data BY query;",
                "queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data)
AS total;",
                "queries_ordered = my_macro_1(queries_count, query);",
                "queries_limit = LIMIT queries_ordered 2;",
                "STORE queries_limit INTO 'output';"
        };
        PigTest test = new PigTest(script, new String[]{});
        test.getPigServer().registerQuery("DEFINE my_macro_1 (QUERY, A) RETURNS C {" +
                "$C = ORDER $QUERY BY total DESC, $A;" +
                "} ;");
        test.assertOutput("data", new String[] {"1"},
                "queries_limit", new String[] {"(1)"});
{code}

> Duplicated macro name error when using pigunit
> ----------------------------------------------
>
>                 Key: PIG-3114
>                 URL: https://issues.apache.org/jira/browse/PIG-3114
>             Project: Pig
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.11
>            Reporter: Chetan Nadgire
>            Assignee: Daniel Dai
>             Fix For: 0.12.0
>
>         Attachments: PIG-3114-2.patch, PIG-3114-3.patch, PIG-3114.patch, PIG-3114.patch,
PatchedPigTest.java, bug_pig.tbz2
>
>
> I'm using PigUnit to test a pig script within which a macro is defined.
> Pig runs fine on cluster but getting parsing error with pigunit.
> So I tried very basic pig script with macro and getting similar error.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing.
<line 9> null. Reason: Duplicated macro name 'my_macro_1'
> 	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> 	at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> 	at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> 	at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:988)
> 	at org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
> 	at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
> 	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> 	at org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
> 	at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:160)
> 	at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:231)
> 	at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:261)
> 	at FirstPigTest.MyPigTest.testTop2Queries(MyPigTest.java:32)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at junit.framework.TestCase.runTest(TestCase.java:176)
> 	at junit.framework.TestCase.runBare(TestCase.java:141)
> 	at junit.framework.TestResult$1.protect(TestResult.java:122)
> 	at junit.framework.TestResult.runProtected(TestResult.java:142)
> 	at junit.framework.TestResult.run(TestResult.java:125)
> 	at junit.framework.TestCase.run(TestCase.java:129)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:255)
> 	at junit.framework.TestSuite.run(TestSuite.java:250)
> 	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> 	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> 	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> Caused by: Failed to parse: <line 9> null. Reason: Duplicated macro name 'my_macro_1'
> 	at org.apache.pig.parser.QueryParserDriver.makeMacroDef(QueryParserDriver.java:406)
> 	at org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:277)
> 	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> 	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> 	... 30 more
>  
> Pig script which is failing :
> {code:title=test.pig|borderStyle=solid}
> DEFINE my_macro_1 (QUERY, A) RETURNS C {
>     $C = ORDER $QUERY BY total DESC, $A;
> } ;
> data =  LOAD 'input' AS (query:CHARARRAY);
> queries_group = GROUP data BY query;
> queries_count = FOREACH queries_group GENERATE group AS query, COUNT(data) AS total;
> queries_ordered = my_macro_1(queries_count, query);
> queries_limit = LIMIT queries_ordered 2;
> STORE queries_limit INTO 'output';
> {code}
> If I remove macro pigunit works fine. Even just defining macro without using it results
in parsing error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message