hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan" <...@yahoo-inc.com>
Subject Revisit Pig Philosophy?
Date Fri, 18 Sep 2009 22:54:51 GMT
Pig Developers,

I looked at the Pig philosophy page as it serves as a guideline for
accepting changes to Pig. Is it time to revisit the overall philosophy? 

Reference: http://hadoop.apache.org/pig/philosophy.html

Some items of interest:

1. SQL semantics and Pig

With the recent addition of SQL on top of Pig, we are making changes to
accommodate SQL semantics. Should this be part of Pig's philosophy?

2. Local mode and other parallel frameworks

<snip>
Pigs Live Anywhere

Pig is intended to be a language for parallel data processing. It is not
tied to one particular parallel framework. It has been implemented first
on hadoop, but we do not intend that to be only on hadoop.
</snip>

Are we still holding onto this? What about local mode? Local mode is not
being treated on equal footing with that of Hadoop for practical
reasons. However, users expect things that work on local mode to work
without any hitches on Hadoop.

Are we still designing the system assuming that Pig will be stacked on
top of other parallel frameworks?

Thanks,
Santhosh

Mime
View raw message