crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chao Shi <stepi...@live.com>
Subject Re: Ability to specify a combiner (with different signature than reducer)
Date Wed, 25 Sep 2013 09:58:00 GMT
Yes. It was a typo. I mean PTable#combineValues.


2013/9/25 Gabriel Reid <gabriel.reid@gmail.com>

> Hi Chao,
>
>
>> Your approach is tricky. I agree that this kind of MR logic is pretty
>> common. So it would be nice to add such feature to crunch. At the first
>> glance, I think the problem in PTable#collectValues is that it return a
>> PTable rather than a PGroupedTable (I haven't check the internal logic yet).
>>
>>
> I think that PTable#collectValues is for a different kind of use case --
> internally it just does a groupByKey and then puts all the values in a
> single collection for each key, so I'm not sure how it would apply here. Or
> did you mean the combineValues method?
>
> - Gabriel
>

Mime
View raw message