crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chandan Biswas <cbiswas1...@gmail.com>
Subject Query on Channels.split
Date Mon, 17 Nov 2014 20:07:23 GMT
Hi,
Currently Channels.split supports only Pair class. But, in our team we need
Tuple support.  Currently what we are doing is -

PCollection<Pair<Pair<A,B>,C> pair1=someDoFn....
Pair<PCollection<Pair<A,B>>,PCollection<C>> pair1AfterFirstSplit =
Channels.split(pair1);
Pair<PCollection<A>,PCollection<B>> pair1AfterSecondSplit =
Channels.split(pair1.first());

instead we can do if Tuple is supported -

PCollection<Tuple3<A,B,C> firstTuple3=someDoFn....
Tuple3<PCollection<A>,PCollection<B>,PCollection<C>>
firstTuple3AfterSplit=firstTuple3.split(...);

Please let me know is if the functionality is already present in Crunch. If
yes, it will be great if anybody directs me to it.

Following the existing pattern in Channels class, I have written separate
Channels classes for Tuple3 and Tuple4. If it's required I can send it.

Thanks,
Chandan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message