macros fail with macro expansion ERROR 1200 when referencing a mount point
---------------------------------------------------------------------------
Key: PIG-2419
URL: https://issues.apache.org/jira/browse/PIG-2419
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.2
Environment: haddoop 0.23.0.1111080202
Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23.0/hadoop-common-project/hadoop-common
-r 1196973p/report.txt /flubber/blue/pigqa/results/pig-0.9.2.1111101150/hadoop-0.23.0.1111080202/report_connecCompiled
by user1 on Tue Nov 8 02:12:04 PST 2011
>From source with checksum 4e42b2d96c899a98a8ab8c7cc23f27ae
-bash-3.1$ pig -version
Apache Pig version 0.9.2.1111101150 (r1200499)
compiled Nov 10 2011, 19:50:15g-0.9.2.1111101150/hadoop-0.23.0.1111080202/report_connection_errs.txt
-bash-3.1$ .txt
Mountside tables are enabled. /data1 is a mount point for /user/user1/pig/tests/data
Reporter: Araceli Henley
DESCRIPTION: Tests are identical except for mount point reference
FAILS:
import '/homes/user2/pigtest/pigtest_next/pigharness/dist/pig_harness/libexec/PigTest/next_macro1.pig';
register /homes/user2/pigtest/pigtest_next/pigharness/dist/pig_harness/lib/java/testudf.jar
result = marcro1 ('/data1/singlefile/studenttab10k', '/data1/singlefile/votertab10k');
store result into '/user/user1/pig/out/user1.1323459597/dotNext_Macros_5.out';
SUCCEEDS:
import '/homes/user2/pigtest/pigtest_next/pigharness/dist/pig_harness/libexec/PigTest/next_macro1.pig';
register /homes/user2/pigtest/pigtest_next/pigharness/dist/pig_harness/lib/java/testudf.jar
result = macro1 ( '/user/user1/pig/tests/data/singlefile/studenttab10k', '/user/user1/pig/tests/data/singlefile/votertab10k'
);
store result into '/user/user1/pig/out/user1.1323459937/dotNext_Macros_1.out';
MACRO: Both scripts reference the same macro
define macro1( in1, in2 ) returns f {
a = load '$in1' using PigStorage() as (name, age, gpa);
b = load '$in2' as (name, age, registration, contributions);
c = cogroup a by (name, age), b by (name, age);
d = filter c by not IsEmpty(a);
e = filter d by not IsEmpty(b);
$f = foreach e generate flatten(a), flatten(b);
};
Pig Stack Trace
---------------
ERROR 1200: <file /grid/2/dev/pigqa/out/pigtest/user1/user1.1323459597/dotNext_Macros_5.pig,
line 3> Cannot expand macro 'marcro1'. Reason: Macro must be defined before expansion.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file
/grid/2/dev/pigqa/out/pigtest/user1/user1.1323459597/dotNext_Macros_5.pig, line 3> Cannot
expand macro 'marcro1'. Reason: Macro must be defined before expansion.
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1639)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1584)
at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:561)
at org.apache.pig.Main.main(Main.java:111)
Caused by: Failed to parse: <file /grid/2/dev/pigqa/out/pigtest/user1/user1.1323459597/dotNext_Macros_5.pig,
line 3> Cannot expand macro 'marcro1'. Reason: Macro must be defined before expansion.
at org.apache.pig.parser.PigMacro.macroInline(PigMacro.java:427)
at org.apache.pig.parser.QueryParserDriver.inlineMacro(QueryParserDriver.java:279)
at org.apache.pig.parser.QueryParserDriver.expandMacro(QueryParserDriver.java:268)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:165)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1631)
... 9 more
================================================================================
HDFS:
-bash-3.1$ hadoop fs -ls /user/user1/pig/tests/data/singlefile/studenttab10k
11/12/09 22:06:47 INFO security.JniBasedUnixGroupsNetgroupMapping: Using JniBasedUnixGroupsNetgroupMapping
for Netgroup resolution
hadioFound 1 items
-rw-r--r-- 1 user1 hdfs 219190 2011-12-07 18:00 /user/user1/pig/tests/data/singlefile/studenttab10k
-bash-3.1$ hadoop fs -ls /user/user1/pig/tests/data/singlefile/votertab10k
11/12/09 22:07:07 INFO security.JniBasedUnixGroupsNetgroupMapping: Using JniBasedUnixGroupsNetgroupMapping
for Netgroup resolution
Found 1 items
-rw-r--r-- 1 user1 hdfs 338211 2011-12-07 18:00 /user/user1/pig/tests/data/singlefile/votertab10k
-bash-3.1$ hadoop fs -ls /data1/singlefile/studenttab10k
11/12/09 22:07:26 INFO security.JniBasedUnixGroupsNetgroupMapping: Using JniBasedUnixGroupsNetgroupMapping
for Netgroup resolution
Found 1 items
-rw-r--r-- 1 user1 hdfs 219190 2011-12-07 18:00 /data1/singlefile/studenttab10k
-bash-3.1$ hadoop fs -ls /data1/singlefile/votertab10k
11/12/09 22:07:42 INFO security.JniBasedUnixGroupsNetgroupMapping: Using JniBasedUnixGroupsNetgroupMapping
for Netgroup resolution
Found 1 items
-rw-r--r-- 1 user1 hdfs 338211 2011-12-07 18:00 /data1/singlefile/votertab10k
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
|