flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Moiz S Jinia <moiz.ji...@gmail.com>
Subject Re: Kafka partitions -> task slots? (keyed stream)
Date Tue, 30 May 2017 15:53:15 GMT
I have just 1 job (that has a ProcessFunction with timers).

You're saying that giving more task slots to my job than the number of
partitions on the source topic is not going to help.

This implies that 1 partition cannot be assigned to more than 1 task slot.
That makes sense as otherwise ordering for a partition would not be
guaranteed.

Thanks.

On Tue, May 30, 2017 at 8:43 PM, Stefan Richter <s.richter@data-artisans.com
> wrote:

> Hi,
>
> it is not restricting the parallelism of your job. Only increasing the
> parallelism of your Job’s sources to more than 5 will not bring any
> improvements. All other operators could still benefit from a higher
> parallelism.
>
> > Am 30.05.2017 um 09:49 schrieb Moiz S Jinia <moiz.jinia@gmail.com>:
> >
> > For a keyed stream (where the key is also the message key in the source
> kafka topic), is the parallelism of the job restricted to the number of
> partitions in the topic?
> >
> > Source topic has 5 partitions, but available task slots are 12. (3 task
> managers each with 4 slots)
> >
> > Moiz
>
>

Mime
View raw message