hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Chauhan <ashutosh.chau...@gmail.com>
Subject Re: How to clone a logical plan ?
Date Fri, 06 Nov 2009 01:34:54 GMT
Thanks, Santhosh for quick response and explaination. Saved few hours of
debugging :)

Ashutosh

On Thu, Nov 5, 2009 at 19:21, Santhosh Srinivasan <sms@yahoo-inc.com> wrote:

> If my memory serves me correctly, the logical plan cloning was
> implemented (by me) for cloning inner plans for foreach. As such, the
> top level plan cloning was never tested and some items are marked as
> TODO (see visit methods for LOLoad, LOStore and LOStream).
>
> If you want to use it as you mention in your test cases, then you need
> to add code for cloning the LOLoad, LOStore, LOStream and LOJoin.
>
> Santhosh
>
>
> -----Original Message-----
> From: Santhosh Srinivasan [mailto:sms@yahoo-inc.com]
> Sent: Thursday, November 05, 2009 4:04 PM
> To: pig-dev@hadoop.apache.org
> Subject: RE: How to clone a logical plan ?
>
> You have hit a bug. I think LOJoin has to be added to
> LogicalPlanCloneHelper.java. Can you file a jira?
>
> Thanks,
> Santhosh
>
> -----Original Message-----
> From: Ashutosh Chauhan [mailto:ashutosh.chauhan@gmail.com]
> Sent: Thursday, November 05, 2009 3:28 PM
> To: pig-dev@hadoop.apache.org
> Subject: How to clone a logical plan ?
>
> Hi,
>
> For our cost based optimizer for a given query plan we need to generate
> alternative query plans and evaluate them based on their estimated cost.
> As a result of that, I want to clone a logical plan. I thought
> LogicalPlanCloner is meant for that, but it doesnt seem to work. I added
> this simple test case in TestLogicalPlanBuilder.java
>
>    public void testLogicalPlanCloneHelper() throws
> CloneNotSupportedException{
>        LogicalPlan lp = buildPlan("C = join ( load 'A') by $0, (load
> 'B') by $0;");
>        LogicalPlanCloner cloner = new LogicalPlanCloner(lp);
>        cloner.getClonedPlan();
>    }
>
> and this fails with the following stacktrace:
>
> java.lang.NullPointerException
>        at
> org.apache.pig.impl.logicalLayer.LOVisitor.visit(LOVisitor.java:171)
>        at
> org.apache.pig.impl.logicalLayer.PlanSetter.visit(PlanSetter.java:63)
>        at
> org.apache.pig.impl.logicalLayer.LOJoin.visit(LOJoin.java:213)
>        at org.apache.pig.impl.logicalLayer.LOJoin.visit(LOJoin.java:45)
>        at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.ja
> va:67)
>        at
> org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.ja
> va:69)
>        at
> org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50)
>        at
> org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
>        at
> org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.getClonedPlan(Lo
> gicalPlanCloneHelper.java:73)
>        at
> org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(Logical
> PlanCloner.java:46)
>        at
> org.apache.pig.test.TestLogicalPlanBuilder.testLogicalPlanCloneHelper(Te
> stLogicalPlanBuilder.java:2110)
>
> I am debugging this, but wanted to ask if I have hit a bug here or if I
> am doing something wrong?
>
> Thanks,
> Ashutosh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message