flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jingsong Li <jingsongl...@gmail.com>
Subject Re: batch range sort support
Date Thu, 23 Apr 2020 04:34:47 GMT
Hi, Benchao,

Glad to see your requirement about range partition.
I have a branch to support range partition: [1]

Can you describe your scene in more detail? What sink did you use for your
jobs? A simple and complete business scenario? This can help the community
judge the importance of the range partition.

[1]https://github.com/JingsongLi/flink/commits/range

Best,
Jingsong Lee

On Thu, Apr 23, 2020 at 12:15 PM Benchao Li <libenchao@gmail.com> wrote:

> Hi,
>
> Currently the sort operator in blink planner is global, which has
> bottleneck if we sort a lot of data.
>
> And I found 'table.exec.range-sort.enabled' config in BatchExecSortRule,
> which makes me very exciting.
> After enabling this config, I found that it's not implemented completely
> now. This config changes the distribution
>  from SINGLETON to range for sort operator, however in BatchExecExchange
> we do not deal with range
> distribution, and will throw UnsupportedOperationException.
>
> My question is,
> 1. Is this config just a mistake when we merge blink into flink, and we
> actually didn't plan to implement this?
> 2. If this is in the plan, then which version may we expect it to be ready?
>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>
>

-- 
Best, Jingsong Lee

Mime
View raw message