Mailing-List: contact dev-help@pig.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@pig.apache.org
Date: Mon, 7 Mar 2011 20:52:59 +0000 (UTC)
From: "Thejas M Nair (JIRA)" <jira@apache.org>
To: pig-dev@hadoop.apache.org
Message-ID: 
 <2001127382.2387.1299531179954.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] Commented: (PIG-1618) Switch to new parser generator
 technology
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/PIG-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003588#comment-13003588 ] 

Thejas M Nair commented on PIG-1618:
------------------------------------

The new parser changes also fixes one behavior seen in earlier versions including 0.8 .
With the changes in trunk - flatten of bag with null schema will result in a null schema.

For example,
in 0.8 -
{code}
grunt> describe g;
g: {group: bytearray,a: {null}}
grunt> f = foreach g generate $0 , flatten(a);
grunt> describe f;
f: {group: bytearray,bytearray}
{code}

in trunk with new parser changes -
{code}
grunt> describe g;
g: {group: bytearray,a: {(null)}}
grunt> f = foreach g generate $0 , flatten(a);
grunt> describe f;
Schema for f unknown.
{code}


> Switch to new parser generator technology
> -----------------------------------------
>
>                 Key: PIG-1618
>                 URL: https://issues.apache.org/jira/browse/PIG-1618
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.8.0
>            Reporter: Alan Gates
>            Assignee: Xuefu Zhang
>             Fix For: 0.9.0
>
>         Attachments: NewParser-1.patch, NewParser-10.patch, NewParser-11.patch, NewParser-12.patch, NewParser-13.2.patch, NewParser-13.patch, NewParser-14.patch, NewParser-15.patch, NewParser-18.patch, NewParser-19.3.patch, NewParser-19.patch, NewParser-2.patch, NewParser-3.patch, NewParser-3.patch, NewParser-4.patch, NewParser-5.patch, NewParser-6.patch, NewParser-7.patch, NewParser-8.patches, NewParser-9.patch, antlr-3.2.jar, javadoc.patch
>
>
> There are many bugs in Pig related to the parser, particularly to bad error messages.  After review of Java CC we feel these will be difficult to address using that tool.  Also, the .jjt files used by JavaCC are hard to understand and maintain.  
> ANTLR is being reviewed as the most likely choice to move to, but other parsers will be reviewed as well.
> This JIRA will act as an umbrella issue for other parser issues.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira