pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1618) Switch to new parser generator technology
Date Mon, 07 Mar 2011 20:52:59 GMT

    [ https://issues.apache.org/jira/browse/PIG-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003588#comment-13003588
] 

Thejas M Nair commented on PIG-1618:
------------------------------------

The new parser changes also fixes one behavior seen in earlier versions including 0.8 .
With the changes in trunk - flatten of bag with null schema will result in a null schema.

For example,
in 0.8 -
{code}
grunt> describe g;
g: {group: bytearray,a: {null}}
grunt> f = foreach g generate $0 , flatten(a);
grunt> describe f;
f: {group: bytearray,bytearray}
{code}

in trunk with new parser changes -
{code}
grunt> describe g;
g: {group: bytearray,a: {(null)}}
grunt> f = foreach g generate $0 , flatten(a);
grunt> describe f;
Schema for f unknown.
{code}



> Switch to new parser generator technology
> -----------------------------------------
>
>                 Key: PIG-1618
>                 URL: https://issues.apache.org/jira/browse/PIG-1618
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.8.0
>            Reporter: Alan Gates
>            Assignee: Xuefu Zhang
>             Fix For: 0.9.0
>
>         Attachments: NewParser-1.patch, NewParser-10.patch, NewParser-11.patch, NewParser-12.patch,
NewParser-13.2.patch, NewParser-13.patch, NewParser-14.patch, NewParser-15.patch, NewParser-18.patch,
NewParser-19.3.patch, NewParser-19.patch, NewParser-2.patch, NewParser-3.patch, NewParser-3.patch,
NewParser-4.patch, NewParser-5.patch, NewParser-6.patch, NewParser-7.patch, NewParser-8.patches,
NewParser-9.patch, antlr-3.2.jar, javadoc.patch
>
>
> There are many bugs in Pig related to the parser, particularly to bad error messages.
 After review of Java CC we feel these will be difficult to address using that tool.  Also,
the .jjt files used by JavaCC are hard to understand and maintain.  
> ANTLR is being reviewed as the most likely choice to move to, but other parsers will
be reviewed as well.
> This JIRA will act as an umbrella issue for other parser issues.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message