flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@apache.org>
Subject Re: grouping and sort grouping with KeySelector
Date Mon, 22 Sep 2014 14:54:28 GMT
Hi Martin,

group sorting is currently only possible with field-index keys and not with
A workaround would be to use a map to convert your DataSet into
Tuple3<YouPojoType, GroupType, SortType> and do a groupBy(1).sortGroup(2,
This is actually also what the KeySelector would do under the hood.

Order.ANY gives the optimizer a bit more freedom to decide whether to sort
ascending or descending. However, I guess this is only relevant in very few


2014-09-22 16:31 GMT+02:00 Martin Neumann <mneumann@spotify.com>:

> Hej,
> The data set I'm working on is quite large so I created a pojo class for it
> to make it less messy. I want do do a simple map, group reduce job. But the
> groups needs to be sorted on a secondary key.
> From what I understand I would get that by doing:
> dataset.groupBy( key1 ).sortGroup( key2 , Order.ASCENDING )
> Is that correct?
> And if so how do I do the same call using a KeySelector. While groupBy can
> take a KeySelector sortGroup cant.
> Also can someone explain to me what Order.ANY is about?
> cheers Martin

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message