hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-512) Expressions in foreach lead to errors
Date Thu, 30 Oct 2008 22:42:44 GMT
Expressions in foreach lead to errors
-------------------------------------

                 Key: PIG-512
                 URL: https://issues.apache.org/jira/browse/PIG-512
             Project: Pig
          Issue Type: Bug
    Affects Versions: types_branch
            Reporter: Santhosh Srinivasan
            Assignee: Santhosh Srinivasan
             Fix For: types_branch


Use of expressions that use the same sub-expressions in foreach lead to translation errors.
This issue is caused due to sharing operators across nested plans. To remedy this issue, logical
operators should be cloned and not shared across plans.

{code}
grunt> a = load 'a' as (x, y, z);
grunt> b = foreach a {
>> exp1 = x + y;
>> exp2 = exp1 + x;
>> generate exp1, exp2;
>> }
grunt> explain b;
2008-10-30 15:38:40,257 [main] WARN  org.apache.pig.PigServer - bytearray is implicitly casted
to double under LOAdd Operator
2008-10-30 15:38:40,258 [main] WARN  org.apache.pig.PigServer - bytearray is implicitly casted
to double under LOAdd Operator
2008-10-30 15:38:40,258 [main] WARN  org.apache.pig.PigServer - bytearray is implicitly casted
to double under LOAdd Operator
Logical Plan:
Store sms-Thu Oct 30 11:27:27 PDT 2008-2609 Schema: {double,double} Type: Unknown
|
|---ForEach sms-Thu Oct 30 11:27:27 PDT 2008-2605 Schema: {double,double} Type: bag
    |   |
    |   Add sms-Thu Oct 30 11:27:27 PDT 2008-2600 FieldSchema: double Type: double
    |   |
    |   |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2606 FieldSchema: double Type: double
    |   |   |
    |   |   |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2598 Projections: [0] Overloaded:
false FieldSchema: x: bytearray Type: bytearray
    |   |       Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
    |   |
    |   |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2607 FieldSchema: double Type: double
    |       |
    |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2599 Projections: [1] Overloaded:
false FieldSchema: y: bytearray Type: bytearray
    |           Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
    |   |
    |   Add sms-Thu Oct 30 11:27:27 PDT 2008-2603 FieldSchema: double Type: double
    |   |
    |   |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2601 Projections:  [*]  Overloaded: false
FieldSchema: double Type: double
    |   |   Input: Add sms-Thu Oct 30 11:27:27 PDT 2008-2600|
    |   |   |---Add sms-Thu Oct 30 11:27:27 PDT 2008-2600 FieldSchema: double Type: double
    |   |       |
    |   |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2598 Projections: [0] Overloaded:
false FieldSchema: x: bytearray Type: bytearray
    |   |       |   Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
    |   |       |
    |   |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2599 Projections: [1] Overloaded:
false FieldSchema: y: bytearray Type: bytearray
    |   |           Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
    |   |
    |   |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2608 FieldSchema: double Type: double
    |       |
    |       |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2602 Projections: [0] Overloaded:
false FieldSchema: x: bytearray Type: bytearray
    |           Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597
    |
    |---Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 Schema: {x: bytearray,y: bytearray,z: bytearray}
Type: bag

2008-10-30 15:38:40,272 [main] ERROR org.apache.pig.impl.plan.OperatorPlan - Attempt to give
operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
multiple outputs.  This operator does not support multiple outputs.
2008-10-30 15:38:40,272 [main] ERROR org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor
- Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
multiple outputs.  This operator does not support multiple outputs.
2008-10-30 15:38:40,273 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException:
Unable to explain alias b [org.apache.pig.impl.plan.VisitorException]
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:235)
        at org.apache.pig.PigServer.compilePp(PigServer.java:731)
        at org.apache.pig.PigServer.explain(PigServer.java:495)
        at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:155)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:193)
        at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:94)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
        at org.apache.pig.Main.main(Main.java:282)
Caused by: org.apache.pig.backend.executionengine.ExecException: org.apache.pig.impl.plan.VisitorException
        ... 8 more
Caused by: org.apache.pig.impl.plan.VisitorException
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:324)
        at org.apache.pig.impl.logicalLayer.LOAdd.visit(LOAdd.java:69)
        at org.apache.pig.impl.logicalLayer.LOAdd.visit(LOAdd.java:29)
        at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:805)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:121)
        at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:40)
        at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
        ... 7 more
Caused by: org.apache.pig.impl.plan.PlanException: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject
multiple outputs.  This operator does not support multiple outputs.
        at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:158)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:89)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:320)
        ... 16 more

2008-10-30 15:38:40,274 [main] ERROR org.apache.pig.tools.grunt.GruntParser - Unable to explain
alias b [org.apache.pig.impl.plan.VisitorException]
2008-10-30 15:38:40,274 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException:
Unable to explain alias b [org.apache.pig.impl.plan.VisitorException]

{code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message