hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: switching to different parser in Pig
Date Sat, 14 Feb 2009 13:16:51 GMT
Not even close.

Take, for example,

1) the problem of using the output of a Pig query as the list of files used
as input.

2) Or running some query in an iterative fashion until convergence is
reached.

3) Or running a pig query, doing a matrix computation on the result and then
running another pig query on the output of the matrix computation.

You can do (1) by running a pig query using an external script and then
downloading the output and expanding that into a pig using a template
expansion and then executing that expanded template.

You can do (2) by having an external script that runs a pig program over and
over again, downloading the results and checking for convergence.

You can do (3) by running one program, then downloading results, computing,
uploading results and running another program.

All of these are maintenance nightmares which would be greatly eased if
pig's semantics could be glued nicely into a good scripting language.
Having access to the AST's would make that pretty easy.

On Sat, Feb 14, 2009 at 4:10 AM, pi song <pi.songs@gmail.com> wrote:

> Due to my limited knowledge, I don't quite understand why building ast from
> outside Pig would be helpful. Isn't Pig Latin already good enough to
> interface to the world?
>



-- 
Ted Dunning, CTO
DeepDyve
4600 Bohannon Drive, Suite 220
Menlo Park, CA 94025
www.deepdyve.com
650-324-0110, ext. 738
858-414-0013 (m)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message