flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: DataStream.partitionCustom() - define parallelism
Date Tue, 19 Jul 2016 09:40:09 GMT
Hi,
it should be possible to set the parallelism on the actual downstream
operation. The partitioning operation is just an intermediate.

Cheers,
Aljoscha

On Mon, 18 Jul 2016 at 20:20 vanekjar <vanek.jaromir@gmail.com> wrote:

> Hi all,
>
> I've got one question considering custom partitioning in DataStream API. Is
> it possible to define/change parallelism when doing 'partitionCustom'
> transformation?
>
> As far as I discovered there is no way how to call 'setParallelism()' on
> the
> 'PartitionTransformation' because it's hidden from the user.
>
> In another thread
> (
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Range-partitioning-td10947.html
> )
> I found a short notice about custom partitioner - 'a partitioning will only
> be valid to the point that you change the parallelism.' So the possibility
> of changing parallelism in the subsequent transformation won't solve my
> problem.
>
> How to make sure that both partitioner and number of partitions are valid
> for subsequent transformations?
>
> Thank you
>
>
>
> --
> View this message in context:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DataStream-partitionCustom-define-parallelism-tp12597.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at
> Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message