hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-158) Rework logical plan
Date Tue, 27 May 2008 18:32:02 GMT

     [ https://issues.apache.org/jira/browse/PIG-158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Santhosh Srinivasan updated PIG-158:
------------------------------------

    Attachment: user_func_and_store.patch

Patch, user_func_and_store.patch fixes the following issues:

1. The parser code connects the inputs of expressions to the actual expressions (e.g.: arguments
to LOUserFunc to LOUserFunc; inputs and outputs of LOBinCond to LOBinCond, etc.) while the
visit methods of these operators also visit the inputs leading to duplicate visits of the
inputs.

The visit methods in LOVisitor have been cleaned up to no-operations.

2. The LOStore logical plan was a dummy plan. It is now a full fledged logical plan that can
be executed.

TODO:

The testStore unit test case in TestMapReduce.java is now throwing an exception in org.apache.pig.impl.physicalLayer.relationalOperators.POForEach.
Earlier it was a Null Pointer Exception.

Now:

{code}
Testcase: testStoreFunction took 0.098 sec
    Caused an ERROR
Unable to store for alias: asdf_id
java.io.IOException: Unable to store for alias: asdf_id
    at org.apache.pig.impl.util.WrappedIOException.wrap(WrappedIOException.java:34)
    at org.apache.pig.PigServer.store(PigServer.java:307)
    at org.apache.pig.PigServer.store(PigServer.java:291)
    at org.apache.pig.test.TestMapReduce.testStoreFunction(TestMapReduce.java:208)
Caused by: org.apache.pig.backend.executionengine.ExecException: org.apache.pig.impl.physicalLayer.relationalOperators.POForEach
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:296)
    at org.apache.pig.PigServer.execute(PigServer.java:462)
    at org.apache.pig.PigServer.store(PigServer.java:305)
    ... 17 more
Caused by: java.lang.ClassCastException: org.apache.pig.impl.physicalLayer.relationalOperators.POForEach
    at org.apache.pig.impl.mapReduceLayer.MRCompiler.compile(MRCompiler.java:188)
    at org.apache.pig.impl.mapReduceLayer.Launcher.launchPig(Launcher.java:67)
    at org.apache.pig.impl.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:26)
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:289)
    ... 19 more
{code}

Earlier:

{code}
Testcase: testStoreFunction took 0.127 sec
	Caused an ERROR
Unable to store for alias: asdf_id
java.io.IOException: Unable to store for alias: asdf_id
	at org.apache.pig.impl.util.WrappedIOException.wrap(WrappedIOException.java:34)
	at org.apache.pig.PigServer.store(PigServer.java:307)
	at org.apache.pig.PigServer.store(PigServer.java:291)
	at org.apache.pig.test.TestMapReduce.testStoreFunction(TestMapReduce.java:208)
Caused by: java.lang.NullPointerException
	at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:697)
	at org.apache.pig.impl.logicalLayer.LOStore.visit(LOStore.java:102)
	at org.apache.pig.impl.logicalLayer.LOStore.visit(LOStore.java:35)
	at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
	at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:234)
	at org.apache.pig.PigServer.compilePp(PigServer.java:513)
	at org.apache.pig.PigServer.execute(PigServer.java:460)
	at org.apache.pig.PigServer.store(PigServer.java:305)
	... 17 more
{code}

> Rework logical plan
> -------------------
>
>                 Key: PIG-158
>                 URL: https://issues.apache.org/jira/browse/PIG-158
>             Project: Pig
>          Issue Type: Sub-task
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: logical_operators.patch, logical_operators_rev_1.patch, logical_operators_rev_2.patch,
logical_operators_rev_3.patch, parser_changes.patch, parser_changes_v1.patch, parser_changes_v2.patch,
parser_changes_v3.patch, parser_changes_v4.patch, ParserErrors.txt, user_func_and_store.patch,
visitorWalker.patch
>
>
> Rework the logical plan in line with http://wiki.apache.org/pig/PigExecutionModel

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message