pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <ga...@hortonworks.com>
Subject Re: Adding a flatten that doesn't throw out the row if you have an empty bag?
Date Sat, 10 Dec 2011 02:56:23 GMT
+1, it seems like people often ask for this.  I also would prefer a new operator.  OUTER_FLATTEN


On Dec 9, 2011, at 5:49 PM, Jonathan Coveney wrote:

> I think this would be nice to have. We could either add a LEFTFLATTEN()
> operator, or add a flag to FLATTEN ie FLATTEN({()},true) or something. I
> personally prefer a new operator, but know that flatten is baked into pig
> so there's that.
> I'm willing to do the work, but would like some guidance on what form it
> should take, and where to look to make the changes.
> Currently you can do
> a = group rel by x;
> b = foreach a generate FLATTEN((IsEmpty(rel) ? null : rel)) as (val:type);
> but this can sometimes result in weird issues, and is just generally clunky
> and feels like something pig should do.

View raw message