hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-161) Rework physical plan
Date Wed, 25 Jun 2008 22:10:46 GMT

     [ https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shravan Matthur Narayanamurthy updated PIG-161:
-----------------------------------------------

    Attachment: foreach_gen_comb.patch

A patch for the foreach genearte issue. Includes both the logical side fix from Santhosh and
the physical side fix from me. Also included with this are minor fixes to get tests working.
I have also rolled exprplan.patch into this. 

Some Main things to note in the patch apart from exprplan.patch:
New operator: NewForEach which has a list of physical plans and incorporates the POGenerate
functionality
Changes to Translator: visit(LOForeach method to build NewForEach instead of foreach-generate,
the use of seen list disregarding walker to process inner plans of foreach to handle the implicit
split case though no implicit case has been tested
Changes to MRCompiler: added a new visitNewForEach method for compiling NewForEach into MR,
Modifications to visitSort & visitDistinct methods to work with NewForEach.
Changes to PhysicalOperator: getNext(Bag) method does accumulation
Changes to POProject: getNext(Bag) changed to handle bag projection
Changes to ConstantExpression to return EOP on second call to the getNext method and its ripple
effects in all test cases.

Status of E2E tests: All previously passing tests pass now. I also included TestEvalPipeline
into the suite. All its test cases except the titleNGrams one pass which inludes join, sort
and distinct. The titleNGrams is a known issue.

> Rework physical plan
> --------------------
>
>                 Key: PIG-161
>                 URL: https://issues.apache.org/jira/browse/PIG-161
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: arithmeticOperators.patch, BinCondAndNegative.patch, CastAndMapLookUp.patch,
exprplan.patch, foreach_gen_comb.patch, incr2.patch, incr3.patch, incr4.patch, incr5.patch,
logToPhyTranslator.patch, missingOps.patch, MRCompilerTests_PlansAndOutputs.txt, Phy_AbsClass.patch,
physicalOps.patch, physicalOps.patch, physicalOps.patch, physicalOps.patch, physicalOps_latest.patch,
POCast.patch, POCast.patch, podistinct.patch, pogenerate.patch, pogenerate.patch, pogenerate.patch,
posort.patch, POUserFuncCorrection.patch, TEST-org.apache.pig.test.TestLocalJobSubmission.txt,
TEST-org.apache.pig.test.TestLogToPhyCompiler.txt, TEST-org.apache.pig.test.TestLogToPhyCompiler.txt,
TEST-org.apache.pig.test.TestMapReduce.txt, TEST-org.apache.pig.test.TestTypeCheckingValidator.txt,
TEST-org.apache.pig.test.TestUnion.txt, translator.patch, translator.patch, translator.patch,
translator.patch
>
>
> This bug tracks work to rework all of the physical operators as described in http://wiki.apache.org/pig/PigTypesFunctionalSpec

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message