pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai" <dai...@gmail.com>
Subject Review Request: Switch to new parser generator technology
Date Thu, 03 Mar 2011 01:16:12 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/459/
-----------------------------------------------------------

Review request for pig, Daniel Dai, thejas, and Xuefu Zhang.


Summary
-------

There are many bugs in Pig related to the parser, particularly to bad error messages. After
review of Java CC we feel these will be difficult to address using that tool. Also, the .jjt
files used by JavaCC are hard to understand and maintain.

ANTLR is being reviewed as the most likely choice to move to, but other parsers will be reviewed
as well.

This JIRA will act as an umbrella issue for other parser issues.


This addresses bug PIG-1618.
    https://issues.apache.org/jira/browse/PIG-1618


Diffs
-----

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/Main.java 1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/PigServer.java 1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/SortInfoSetter.java PRE-CREATION

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/StandAloneParser.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/HExecutionEngine.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRCompiler.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/LogToPhyTranslationVisitor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POMergeJoin.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/LOCogroup.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/LOJoin.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/ProjectFixerUpper.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/optimizer/PushDownForeachFlatten.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/optimizer/PushUpFilter.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/schema/Schema.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/plan/OperatorPlan.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/util/MultiMap.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/BaseOperatorPlan.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/OperatorPlan.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/OperatorSubPlan.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/LogicalExpPlanMigrationVistor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/LogicalPlanMigrationVistor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/Util.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/BinCondExpression.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ConstantExpression.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/MapLookupExpression.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/UserFuncExpression.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/AllExpressionVisitor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOCogroup.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOGenerate.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOInnerLoad.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOJoin.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOStore.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOUnion.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogToPhyTranslationVisitor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalPlan.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalRelationalOperator.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalSchema.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/ColumnPruneHelper.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/ConstExpEvaluator.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/ImplicitSplitInserter.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/CastLineageSetter.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/ColumnAliasConversionVisitor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/LineageFindRelVisitor.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/ProjectStarExpander.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/ScalarVisitor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/TypeCheckingExpVisitor.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/TypeCheckingRelVisitor.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/visitor/UnionOnSchemaSetter.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/AstValidator.g 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/FunctionType.java PRE-CREATION

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/InvalidScalarProjectionException.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanBuilder.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/LogicalPlanGenerator.g
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/ParserException.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/ParserValidationException.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryLexer.g 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryParser.g 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryParserDriver.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/QueryParserUtils.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/parser/StreamingCommandUtils.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/pen/AugmentBaseDataVisitor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/pen/ExampleGenerator.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/grunt/GruntParser.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/PigStats.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/PigStatsUtil.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/ScriptState.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/pigstats/SimplePigStats.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/ParserTestingUtils.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestAstValidator.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestColumnAliasConversion.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestLexer.pig 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestLogicalPlanGenerator.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestParser.pig 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestQueryLexer.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestQueryParser.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/parser/TestScalarVisitor.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestCollectedGroup.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestDataBagAccess.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestEvalPipeline.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestEvalPipeline2.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestEvalPipelineLocal.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFRJoin.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestForEachStar.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestGrunt.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestJoin.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestJoinSmoke.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestLogToPhyCompiler.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMRCompiler.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMergeJoin.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQuery.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQueryBasic.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQueryCompiler.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMultiQueryLocal.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanFilterRule.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanListener.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanLogicalOptimizer.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPlanOperatorPlan.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNullConstant.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigRunner.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigServer.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigStats.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPinOptions.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPruneColumn.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestScalarAliases.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestSchema.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestSkewedJoin.java 1076316

  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestTypeCheckingValidatorNewLP.java
PRE-CREATION 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestUnionOnSchema.java
1076316 
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/utils/TypeCheckingTestUtil.java
1076316 

Diff: https://reviews.apache.org/r/459/diff


Testing
-------


Thanks,

Daniel


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message