hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-512) Expressions in foreach lead to errors
Date Thu, 13 Nov 2008 19:46:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Santhosh Srinivasan updated PIG-512:
------------------------------------

    Patch Info: [Patch Available]

> Expressions in foreach lead to errors
> -------------------------------------
>
>                 Key: PIG-512
>                 URL: https://issues.apache.org/jira/browse/PIG-512
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Santhosh Srinivasan
>            Assignee: Santhosh Srinivasan
>             Fix For: types_branch
>
>         Attachments: PIG-512.patch
>
>
> Use of expressions that use the same sub-expressions in foreach lead to translation errors.
This issue is caused due to sharing operators across nested plans. To remedy this issue, logical
operators should be cloned and not shared across plans.
> {code}
> grunt> a = load 'a' as (x, y, z);
> grunt> b = foreach a {
> >> exp1 = x + y;
> >> exp2 = exp1 + x;
> >> generate exp1, exp2;
> >> }
> grunt> explain b;
> 2008-10-30 15:38:40,257 [main] WARN  org.apache.pig.PigServer - bytearray is implicitly
casted to double under LOAdd Operator
> 2008-10-30 15:38:40,258 [main] WARN  org.apache.pig.PigServer - bytearray is implicitly
casted to double under LOAdd Operator
> 2008-10-30 15:38:40,258 [main] WARN  org.apache.pig.PigServer - bytearray is implicitly
casted to double under LOAdd Operator
> Logical Plan:
> Store sms-Thu Oct 30 11:27:27 PDT 2008-2609 Schema: {double,double} Type: Unknown
> |
> |---ForEach sms-Thu Oct 30 11:27:27 PDT 2008-2605 Schema: {double,double} Type: bag
>     |   |
>     |   Add sms-Thu Oct 30 11:27:27 PDT 2008-2600 FieldSchema: double Type: double
>     |   |
>     |   |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2606 FieldSchema: double Type: double
>     |   |   |
>     |   |   |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2598 Projections: [0] Overloaded:
false FieldSchema: x: bytearray Type: bytearray
>     |   |       Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
>     |   |
>     |   |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2607 FieldSchema: double Type: double
>     |       |
>     |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2599 Projections: [1] Overloaded:
false FieldSchema: y: bytearray Type: bytearray
>     |           Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
>     |   |
>     |   Add sms-Thu Oct 30 11:27:27 PDT 2008-2603 FieldSchema: double Type: double
>     |   |
>     |   |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2601 Projections:  [*]  Overloaded:
false FieldSchema: double Type: double
>     |   |   Input: Add sms-Thu Oct 30 11:27:27 PDT 2008-2600|
>     |   |   |---Add sms-Thu Oct 30 11:27:27 PDT 2008-2600 FieldSchema: double Type: double
>     |   |       |
>     |   |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2598 Projections: [0] Overloaded:
false FieldSchema: x: bytearray Type: bytearray
>     |   |       |   Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
>     |   |       |
>     |   |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2599 Projections: [1] Overloaded:
false FieldSchema: y: bytearray Type: bytearray
>     |   |           Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
>     |   |
>     |   |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2608 FieldSchema: double Type: double
>     |       |
>     |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2602 Projections: [0] Overloaded:
false FieldSchema: x: bytearray Type: bytearray
>     |           Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
>     |
>     |---Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 Schema: {x: bytearray,y: bytearray,z:
bytearray} Type: bag
> 2008-10-30 15:38:40,272 [main] ERROR org.apache.pig.impl.plan.OperatorPlan - Attempt
to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
multiple outputs.  This operator does not support multiple outputs.
> 2008-10-30 15:38:40,272 [main] ERROR org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor
- Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
multiple outputs.  This operator does not support multiple outputs.
> 2008-10-30 15:38:40,273 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException:
Unable to explain alias b [org.apache.pig.impl.plan.VisitorException]
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:235)
>         at org.apache.pig.PigServer.compilePp(PigServer.java:731)
>         at org.apache.pig.PigServer.explain(PigServer.java:495)
>         at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:155)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:193)
>         at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:94)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>         at org.apache.pig.Main.main(Main.java:282)
> Caused by: org.apache.pig.backend.executionengine.ExecException: org.apache.pig.impl.plan.VisitorException
>         ... 8 more
> Caused by: org.apache.pig.impl.plan.VisitorException
>         at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:324)
>         at org.apache.pig.impl.logicalLayer.LOAdd.visit(LOAdd.java:69)
>         at org.apache.pig.impl.logicalLayer.LOAdd.visit(LOAdd.java:29)
>         at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
>         at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:805)
>         at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:121)
>         at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:40)
>         at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
>         at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
>         ... 7 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
multiple outputs.  This operator does not support multiple outputs.
>         at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:158)
>         at org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:89)
>         at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:320)
>         ... 16 more
> 2008-10-30 15:38:40,274 [main] ERROR org.apache.pig.tools.grunt.GruntParser - Unable
to explain alias b [org.apache.pig.impl.plan.VisitorException]
> 2008-10-30 15:38:40,274 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException:
Unable to explain alias b [org.apache.pig.impl.plan.VisitorException]
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message