flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Question about the custom partitioner
Date Fri, 16 Jun 2017 09:19:01 GMT

I’m afraid that’s not possible out-of-box with the current APIs. I actually don’t know
why the user-facing Partitioner only allows returning one target because the internal StreamPartitioner
(which extends ChannelSelector) allows returning multiple target partitions.

You can hack around the API by manually creating your own StreamPartitioner and applying it
to the DataStream as DataStream.partitionCustom() and DataStream.setConnectionType() (the
first calls the latter) do.


> On 14. Jun 2017, at 09:09, Xingcan Cui <xingcanc@gmail.com> wrote:
> Hi all,
> I want to duplicate records to multiple downstream tasks (not all of them thus the
> Broadcasting should not work) in stream environment.
> However, it seems that the current custom partitioner can return only one partition index.
> Why this restriction exists or do I miss something?
> Thanks,
> Xingcan

View raw message