pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang" <xue...@yahoo-inc.com>
Subject Re: Review Request: Switch to new parser generator technology
Date Thu, 03 Mar 2011 18:20:58 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/459/#review292
-----------------------------------------------------------



http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/PigServer.java
<https://reviews.apache.org/r/459/#comment547>

    this member variable is not used. Need to find out if we still need it.



http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/PigServer.java
<https://reviews.apache.org/r/459/#comment548>

    markAsExecuted() is currently no-op. Needs to find out if we still need it.



http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java
<https://reviews.apache.org/r/459/#comment546>

    option pig.usenewlogicalplan is no longer needed. I have removed the from the code.



http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java
<https://reviews.apache.org/r/459/#comment549>

    Here some logic is duplicated with that in ColumnAliasConversionVisitor. Need to find
out if we need in both places.



http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalPlan.java
<https://reviews.apache.org/r/459/#comment550>

    We may remove this one and provide an alternative for test cases.



http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalSchema.java
<https://reviews.apache.org/r/459/#comment551>

    Here we use endsWith(). I'm wondering what happens when user gives X::a where X isn't
a valid relation alias.



http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigRunner.java
<https://reviews.apache.org/r/459/#comment544>

    TODO item needs to be filled in.



http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestSchema.java
<https://reviews.apache.org/r/459/#comment545>

    Is till testcase still valid?



http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestTypeCheckingValidatorNewLP.java
<https://reviews.apache.org/r/459/#comment552>

    There are several test cases that are currently commented out. We need to enable them
if they are indeed valid.


I have reviewed all changes except for a few new files. Many small, obvious item that I found
have been fixed in the code so I didn't list them here.

- Xuefu


On 2011-03-02 17:16:11, Daniel Dai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/459/
> -----------------------------------------------------------
> 
> (Updated 2011-03-02 17:16:11)
> 
> 
> Review request for pig, Daniel Dai, thejas, and Xuefu Zhang.
> 
> 
> Summary
> -------
> 
> There are many bugs in Pig related to the parser, particularly to bad error messages.
After review of Java CC we feel these will be difficult to address using that tool. Also,
the .jjt files used by JavaCC are hard to understand and maintain.
> 
> ANTLR is being reviewed as the most likely choice to move to, but other parsers will
be reviewed as well.
> 
> This JIRA will act as an umbrella issue for other parser issues.
> 
> 
> This addresses bug PIG-1618.
>     https://issues.apache.org/jira/browse/PIG-1618
> 
> 
> Diffs
> -----
> 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/Main.java 1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/PigServer.java 1076316

>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/SortInfoSetter.java PRE-CREATION

>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/StandAloneParser.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRCompiler.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/LogToPhyTranslationVisitor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POMergeJoin.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/LOCogroup.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/LOJoin.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/ProjectFixerUpper.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/optimizer/PushDownForeachFlatten.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/optimizer/PushUpFilter.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/schema/Schema.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/plan/OperatorPlan.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/util/MultiMap.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/BaseOperatorPlan.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/OperatorPlan.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/OperatorSubPlan.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/LogicalExpPlanMigrationVistor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/LogicalPlanMigrationVistor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/Util.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/BinCondExpression.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ConstantExpression.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/MapLookupExpression.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/UserFuncExpression.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/AllExpressionVisitor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOCogroup.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOGenerate.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOInnerLoad.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOJoin.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOStore.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOUnion.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogToPhyTranslationVisitor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalPlan.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalRelationalOperator.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalSchema.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/ColumnPruneHelper.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/ConstExpEvaluator.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/ImplicitSplitInserter.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/CastLineageSetter.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/ColumnAliasConversionVisitor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/LineageFindRelVisitor.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/ProjectStarExpander.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/ScalarVisitor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/TypeCheckingExpVisitor.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/TypeCheckingRelVisitor.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/UnionOnSchemaSetter.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/AstValidator.g
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/FunctionType.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/InvalidScalarProjectionException.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanGenerator.g
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/ParserException.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/ParserValidationException.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryLexer.g 1076316

>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryParser.g 1076316

>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryParserDriver.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryParserUtils.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/StreamingCommandUtils.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/pen/AugmentBaseDataVisitor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/pen/ExampleGenerator.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/grunt/GruntParser.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/PigStats.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/PigStatsUtil.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/ScriptState.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/SimplePigStats.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/ParserTestingUtils.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestAstValidator.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestColumnAliasConversion.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestLexer.pig
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestLogicalPlanGenerator.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestParser.pig
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestQueryLexer.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestQueryParser.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestScalarVisitor.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestCollectedGroup.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestDataBagAccess.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestEvalPipeline.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestEvalPipeline2.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestEvalPipelineLocal.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFRJoin.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestForEachStar.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestGrunt.java 1076316

>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestJoin.java 1076316

>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestJoinSmoke.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestLogToPhyCompiler.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMRCompiler.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMergeJoin.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQuery.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQueryBasic.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQueryCompiler.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQueryLocal.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanFilterRule.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanListener.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanLogicalOptimizer.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanOperatorPlan.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNullConstant.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigRunner.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigServer.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigStats.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPinOptions.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPruneColumn.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestScalarAliases.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestSchema.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestSkewedJoin.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestTypeCheckingValidatorNewLP.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestUnionOnSchema.java
1076316 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/utils/TypeCheckingTestUtil.java
1076316 
> 
> Diff: https://reviews.apache.org/r/459/diff
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Daniel
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message