crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <josh.wi...@gmail.com>
Subject Re: Query on Channels.split
Date Mon, 17 Nov 2014 23:27:07 GMT
Hey Chandan,

No, it doesn't exist right now, please do send it along.

Josh
On Mon, Nov 17, 2014 at 12:07 PM Chandan Biswas <cbiswas1983@gmail.com>
wrote:

> Hi,
> Currently Channels.split supports only Pair class. But, in our team we need
> Tuple support.  Currently what we are doing is -
>
> PCollection<Pair<Pair<A,B>,C> pair1=someDoFn....
> Pair<PCollection<Pair<A,B>>,PCollection<C>> pair1AfterFirstSplit
=
> Channels.split(pair1);
> Pair<PCollection<A>,PCollection<B>> pair1AfterSecondSplit =
> Channels.split(pair1.first());
>
> instead we can do if Tuple is supported -
>
> PCollection<Tuple3<A,B,C> firstTuple3=someDoFn....
> Tuple3<PCollection<A>,PCollection<B>,PCollection<C>>
> firstTuple3AfterSplit=firstTuple3.split(...);
>
> Please let me know is if the functionality is already present in Crunch. If
> yes, it will be great if anybody directs me to it.
>
> Following the existing pattern in Channels class, I have written separate
> Channels classes for Tuple3 and Tuple4. If it's required I can send it.
>
> Thanks,
> Chandan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message