pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Ryaboy <dvrya...@gmail.com>
Subject Re: Sometimes-Algebraic functions
Date Tue, 30 Aug 2011 21:24:41 GMT
Wow this is so much less trivial than I thought it was going to be! :)

D

On Mon, Aug 29, 2011 at 9:36 PM, Ashutosh Chauhan <hashutosh@apache.org>wrote:

> Yup, that makes sense. Class returned by argToFuncMapping needs to
> implement
> Algebraic instead of EvalFunc itself.
>
> Ashutosh
> On Mon, Aug 29, 2011 at 15:46, Dmitriy Ryaboy <dvryaboy@gmail.com> wrote:
>
> > Had a fun discovery today: a user tried to do the following (perfectly
> > reasonable) thing:
> >
> > bag_sizes = foreach grouped_data genrate
> >  group, SIZE(grouped_data);
> >
> > .. and it was excessively slow, because SIZE is not algebraic.
> >
> > I had him switch to COUNT_STAR, and that worked, of course.
> >
> > But he reasonably pointed out that SIZE should work, too. It's documented
> > to
> > work on bags, after all.
> >
> > I tried switching SIZE's argToFunc mapping to return
> > COUNT_STAR.class.getName(), but discovered much to my dismay that
> algebraic
> > optimization still does not get invoked.
> >
> > Presumably because we check if SIZE is instanceof Algebraic, instead of
> > checking the class that's returned by argToFuncMapping.
> >
> > Is that about right? Do you guys agree that's a bug?
> >
> > D
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message