pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <ga...@yahoo-inc.com>
Subject Re: switching to different parser in Pig
Date Mon, 23 Feb 2009 16:36:32 GMT
We looked into antlr.  It appears to be very similar to javacc, with  
the added feature that the java code it generates is humanly  
readable.  That isn't why we want to switch off of javacc.  Olga  
listed the 3 things we want out of a parser that javacc isn't giving  
us (lack of docs, no easy customization of error handle, decoupling of  
scanning and parsing).  So antlr doesn't look viable.

In response to Pi's suggestion that we could use the logical plan, I  
hope we could use something close to it.  Whatever we choose we want  
it to be flexible enough to represent richer language constructs (like  
branch and loop).  I'm not sure our current logical plan can do that.   
At the same time, we don't need another layer of translation (we  
already have logical -> physical -> mapreduce).  I would like to find  
a representation that could handle expressing the syntax and what is  
currently the logical plan.

Alan.

On Feb 20, 2009, at 5:15 PM, pi song wrote:

> Should be pretty close but we may need to cleanup the interface a  
> bit. Then
> the new parser  module can be switched in easily.
> BTW, have we already got the solution for the new parser generator?
>
> Pi
>
>
> On Fri, Feb 20, 2009 at 9:03 PM, Ted Dunning <ted.dunning@gmail.com>  
> wrote:
>
>>
>> Probably nearly the same effect as you suggest.  Are the concepts  
>> at the
>> logical plan layer similar to those expressed in pig latin?  Or has a
>> significant transformation occurred by then?
>>
>>
>> On Fri, Feb 20, 2009 at 1:59 AM, pi song <pi.songs@gmail.com> wrote:
>>
>>> Sounds good but how about exposing the logical plan layer instead?
>>> Wouldn't
>>> that yield the same effect?  From python for example you still can
>>> construct
>>> a logical plan and give to Pig to execute.
>>>
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
>>


Mime
View raw message