flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyula.f...@gmail.com>
Subject Re: How to configure max parallelism?
Date Wed, 25 Jan 2017 10:13:16 GMT
Great, thank you for the explanation Ufuk :)

Gyula

Ufuk Celebi <uce@apache.org> ezt írta (időpont: 2017. jan. 25., Sze, 11:11):

> Hey Gyula,
>
> as far as I can tell, there are no docs for this yet. Good news is
> that the docs are getting a lot of love for the release and I guess
> this is simply something that is still missing.
>
> The setMaxParallelism call should be available in all the places where
> you can also call setParallelism. You have the option to set it on the
> environment and per operator.
>
> If you don't configure anything, we compute somewhat sane defaults in
> order to allow scaling up later:
> Parallelism 1-64 => default max parallelism: 128
> Parallelism 65+ => parallelism + parallelism/2 (where I parallelism is
> rounded up to the next power of two)
>
> The maximum allowed max parallelism is 32768.
>
> The computation happens here:
>
> https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/KeyGroupRangeAssignment.java#L126
> .
>
> – Ufuk
>
>
> On Wed, Jan 25, 2017 at 9:05 AM, Gyula Fóra <gyula.fora@gmail.com> wrote:
> > Hi all,
> >
> > I can't seem to find in the documentation how to set the maximum
> parallelism
> > for rescaling keyed state.
> >
> > Can anyone help me out here?
> >
> > Thanks!
> > Gyula
>

Mime
View raw message