hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "pi song" <pi.so...@gmail.com>
Subject Re: OLAP ?
Date Tue, 01 Jul 2008 13:05:45 GMT
We will run a combiner in map phase soon right? Potentially I can just take
advantage of that.

On Tue, Jul 1, 2008 at 7:35 AM, Olga Natkovich <olgan@yahoo-inc.com> wrote:

> Pi,
>
> This is a great start!
>
> Looked a little but into proposed implementation - seems like we should
> be able to do a  lot of hash-based preaggregation in the map and then
> sort based preaggregation in the combiner.
>
> Olga
>
> > -----Original Message-----
> > From: Chris Olston [mailto:olston@yahoo-inc.com]
> > Sent: Monday, June 30, 2008 8:49 AM
> > To: pig-dev@incubator.apache.org
> > Subject: Re: OLAP ?
> >
> > Awesome idea, Pi!
> >
> > Bottom line: sooner or later somebody is going to package up
> > a "olap- on-mapreduce" solution. Pig seems like a very good
> > vehicle for it.
> >
> > -Chris
> >
> >
> > On Jun 30, 2008, at 8:10 AM, pi song wrote:
> >
> > > Has anyone ever looked at OLAP Cube processing on Pig or
> > MapReduce?
> > > This feature should be beneficial for normal businesses
> > (probably not
> > > web scale).  I find it pretty simple provided that we can work on
> > > relational abstraction (but might not scale well on really big
> > > datasets). I have done a sketch on how I think it should look like
> > > here http://wiki.apache.org/pig/PigOLAPSketch . Welcome for all
> > > comments/suggestions.
> > >
> > > Pi
> >
> > --
> > Christopher Olston, Ph.D.
> > Sr. Research Scientist
> > Yahoo! Research
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message