hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-158) Rework logical plan
Date Sat, 19 Apr 2008 06:08:21 GMT

    [ https://issues.apache.org/jira/browse/PIG-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12590678#action_12590678
] 

Santhosh Srinivasan commented on PIG-158:
-----------------------------------------

I have posted a patch for the parser changes to use the new Logical Operators. I had to change
some of the logical operators in order to use the information available at parse time. This
is a work in progress. The test coverage is about 66%. I am blocked on three issues.


test:
   [delete] Deleting directory /mnt/127.0.0.1/home/sms/src_pig/pig/branches/types/build/test/logs
    [mkdir] Created dir: /mnt/127.0.0.1/home/sms/src_pig/pig/branches/types/build/test/logs
    [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
    [junit] Tests run: 48, Failures: 15, Errors: 1, Time elapsed: 0.375 sec
    [junit] Test org.apache.pig.test.TestLogicalPlanBuilder FAILED

>From the test run there are 48 test cases, 32 were successful, 15 failed and there was
1 Error:

1 Error: Exectype.LOCAL in PigContext is commented out

15 Failures: The failures are categorized into three classes

i. Unable to instantiate function using pigContextInstantiateFunc
---------------------------------------------------------------------------------------

Eg:

foreach (cogroup (load 'a') by $1, (load 'b') by $1) generate org.apache.pig.builtin.AVG($1)
;


ii. Not able to resolve field positions in nested operators in a foreach clause
-------------------------------------------------------------------------------------------------------

Eg:

foreach (load 'a') {B = FILTER $0 BY (($1 == $2) AND ('a' < 'b'));generate B;};
                               ^^^

iii. LOProject v/s LOGenerate
----------------------------------------

The expression A.x can be used in two contexts

1. foreach C {B = A.x; generate B;}
2. foreach C {generate A.x;}

Statement 1 results in a LOGenerate for B = A.x whereas in statement 2, A.x results in a LOProject.
A suggestion is to disallow assignments of the form B = A.x.

There is a bug with the schema generation in LOCogroup which is leading to errors in statement
2.


Any thoughts/suggestions/questions/comments are welcome.

Thanks,
Santhosh


> Rework logical plan
> -------------------
>
>                 Key: PIG-158
>                 URL: https://issues.apache.org/jira/browse/PIG-158
>             Project: Pig
>          Issue Type: Sub-task
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: logical_operators.patch, logical_operators_rev_1.patch, logical_operators_rev_2.patch,
logical_operators_rev_3.patch, visitorWalker.patch
>
>
> Rework the logical plan in line with http://wiki.apache.org/pig/PigExecutionModel

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message