hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Olston (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-730) problem combining schema from a union of several LOAD expressions, with a nested bag inside the schema.
Date Tue, 24 Mar 2009 20:17:50 GMT

     [ https://issues.apache.org/jira/browse/PIG-730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Christopher Olston updated PIG-730:
-----------------------------------

    Description: 
grunt> a = load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)});
grunt> b = union (load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})),
(load 'bar' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)}));
grunt> c = foreach a generate flatten(outlinks.target);
grunt> d = foreach b generate flatten(outlinks.target);

---> Would expect both C and D to work, but only C works. D gives the following error:
---> Turns out using outlinks.t.target works for D but not for C.
---> I don't care which one, but the same syntax should work for both!

2009-03-24 13:15:05,376 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error
during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
Details at logfile: /echo/olston/data/pig_1237925683748.log
grunt> quit

$ cat pig_1237925683748.log 
ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Invalid
alias: target in {t: (target: chararray,text: chararray)}
        at org.apache.pig.PigServer.parseQuery(PigServer.java:317)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:276)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:529)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:280)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
        at org.apache.pig.Main.main(Main.java:321)
Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: Invalid alias: target in
{t: (target: chararray,text: chararray)}
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.AliasFieldOrSpec(QueryParser.java:6042)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.ColOrSpec(QueryParser.java:5898)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.BracketedSimpleProj(QueryParser.java:5423)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:4100)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:3967)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:3920)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:3829)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:3755)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:3721)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItem(QueryParser.java:3617)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItemList(QueryParser.java:3557)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.GenerateStatement(QueryParser.java:3514)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.java:2985)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:2395)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1028)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:804)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:595)
        at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:60)
        at org.apache.pig.PigServer.parseQuery(PigServer.java:310)
        ... 6 more


  was:
links = (union (load '/echo/olston/data/Echo-fetcher_first_extractedLinks-1' using BinStorage
as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})), (load '/echo/olston/data/Echo-fetcher_first_extractedLinks-2'
using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})));
DESCRIBE links;

gives this error:

2009-03-24 12:33:59,657 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot
cast bag with schema outlinks: bag({t: (target: chararray,text: chararray)}) to tuple with
schema tuple
Details at logfile: /echo/olston/data/pig_1237923219665.log

the pig_...log file contains:

ERROR 1052: Cannot cast bag with schema outlinks: bag({t: (target: chararray,text: chararray)})
to tuple with schema tuple
org.apache.pig.impl.logicalLayer.FrontendException: Unable to describe schema for alias links
        at org.apache.pig.PigServer.dumpSchema(PigServer.java:394)
        at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:153)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:188)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:72)
        at org.apache.pig.Main.main(Main.java:296)
Caused by: org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused
the validation to stop
        at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
        at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
        at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
        at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:79)
        at org.apache.pig.PigServer.compileLp(PigServer.java:722)
        at org.apache.pig.PigServer.dumpSchema(PigServer.java:386)
        ... 5 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: Problem while
casting inputs of Union
        at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2143)
        at org.apache.pig.impl.logicalLayer.LOUnion.visit(LOUnion.java:113)
        at org.apache.pig.impl.logicalLayer.LOUnion.visit(LOUnion.java:34)
        at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
        ... 10 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: Cannot cast bag
with schema outlinks: bag({t: (target: chararray,text: chararray)}) to tuple with schema tuple
        at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2034)
        at org.apache.pig.impl.logicalLayer.LOCast.visit(LOCast.java:65)
        at org.apache.pig.impl.logicalLayer.LOCast.visit(LOCast.java:29)
        at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.checkInnerPlan(TypeCheckingVisitor.java:2944)
        at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2849)
        at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2137)
        ... 15 more



> problem combining schema from a union of several LOAD expressions, with a nested bag
inside the schema.
> -------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-730
>                 URL: https://issues.apache.org/jira/browse/PIG-730
>             Project: Pig
>          Issue Type: Bug
>         Environment: pig local mode
>            Reporter: Christopher Olston
>
> grunt> a = load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)});
> grunt> b = union (load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})),
(load 'bar' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)}));
> grunt> c = foreach a generate flatten(outlinks.target);
> grunt> d = foreach b generate flatten(outlinks.target);
> ---> Would expect both C and D to work, but only C works. D gives the following error:
> ---> Turns out using outlinks.t.target works for D but not for C.
> ---> I don't care which one, but the same syntax should work for both!
> 2009-03-24 13:15:05,376 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error
during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
> Details at logfile: /echo/olston/data/pig_1237925683748.log
> grunt> quit
> $ cat pig_1237925683748.log 
> ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text:
chararray)}
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing.
Invalid alias: target in {t: (target: chararray,text: chararray)}
>         at org.apache.pig.PigServer.parseQuery(PigServer.java:317)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:276)
>         at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:529)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:280)
>         at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>         at org.apache.pig.Main.main(Main.java:321)
> Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: Invalid alias: target
in {t: (target: chararray,text: chararray)}
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.AliasFieldOrSpec(QueryParser.java:6042)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.ColOrSpec(QueryParser.java:5898)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.BracketedSimpleProj(QueryParser.java:5423)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:4100)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:3967)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:3920)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:3829)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:3755)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:3721)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItem(QueryParser.java:3617)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItemList(QueryParser.java:3557)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.GenerateStatement(QueryParser.java:3514)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.java:2985)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:2395)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1028)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:804)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:595)
>         at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:60)
>         at org.apache.pig.PigServer.parseQuery(PigServer.java:310)
>         ... 6 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message