+1, it seems like people often ask for this. I also would prefer a new operator. OUTER_FLATTEN
maybe?
Alan.
On Dec 9, 2011, at 5:49 PM, Jonathan Coveney wrote:
> I think this would be nice to have. We could either add a LEFTFLATTEN()
> operator, or add a flag to FLATTEN ie FLATTEN({()},true) or something. I
> personally prefer a new operator, but know that flatten is baked into pig
> so there's that.
>
> I'm willing to do the work, but would like some guidance on what form it
> should take, and where to look to make the changes.
>
> Currently you can do
>
> a = group rel by x;
> b = foreach a generate FLATTEN((IsEmpty(rel) ? null : rel)) as (val:type);
>
> but this can sometimes result in weird issues, and is just generally clunky
> and feels like something pig should do.
|