flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chakravarthy varaga <chakravarth...@gmail.com>
Subject Re: Increasing parallelism skews/increases overall job processing time linearly
Date Sat, 07 Jan 2017 13:35:40 GMT
Hi Stephen

. Kafka version is: 0.9.0.1 the connector is flinkconsumer09
. The flatmap n coflatmap are connected by keyBy
. No data is broadcasted and the data is not exploded based on the
parallelism

Cvp

On 6 Jan 2017 20:16, "Stephan Ewen" <sewen@apache.org> wrote:

> Hi!
>
> You are right, parallelism 2 should be faster than parallelism 1 ;-) As
> ChenQin pointed out, having only 2 Kafka Partitions may prevent further
> scaleout.
>
> Few things to check:
>   - How are you connecting the FlatMap and CoFlatMap? Default, keyBy,
> broadcast?
>   - Broadcast for example would multiply the data based on parallelism,
> can lead to slowdown when saturating the network.
>   - Are you using the standard Kafka Source (which Kafka version)?
>   - Is there any part in the program that multiplies data/effort with
> higher parallelism (does the FlatMap explode data based on parallelism)?
>
> Stephan
>
>
> On Fri, Jan 6, 2017 at 7:27 PM, Chen Qin <qinnchen@gmail.com> wrote:
>
>> Just noticed there are only two partitions per topic. Regardless of how
>> large parallelism set. Only two of those will get partition assigned at
>> most.
>>
>> Sent from my iPhone
>>
>> On Jan 6, 2017, at 02:40, Chakravarthy varaga <chakravarthyvp@gmail.com>
>> wrote:
>>
>> Hi All,
>>
>>     Any updates on this?
>>
>> Best Regards
>> CVP
>>
>> On Thu, Jan 5, 2017 at 1:21 PM, Chakravarthy varaga <
>> chakravarthyvp@gmail.com> wrote:
>>
>>>
>>> Hi All,
>>>
>>> I have a job as attached.
>>>
>>> I have a 16 Core blade running RHEL 7. The taskmanager default number of
>>> slots is set to 1. The source is a kafka stream and each of the 2
>>> sources(topic) have 2 partitions each.
>>>
>>>
>>> *What I notice is that when I deploy a job to run with #parallelism=2
>>> the total processing time doubles the time it took when the same job was
>>> deployed with #parallelism=1. It linearly increases with the parallelism.*
>>> Since the numberof slots is set to 1 per TM, I would assume that the job
>>> would be processed in parallel in 2 different TMs and that each consumer in
>>> each TM is connected to 1 partition of the topic. This therefore should
>>> have kept the overall processing time the same or less !!!
>>>
>>> The co-flatmap connects the 2 streams & uses ValueState (checkpointed in
>>> FS). I think this is distributed among the TMs. My understanding is that
>>> the search of values state could be costly between TMs.  Do you sense
>>> something wrong here?
>>>
>>> Best Regards
>>> CVP
>>>
>>>
>>>
>>>
>>>
>>
>

Mime
View raw message