flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Performance insights
Date Fri, 05 Feb 2016 14:34:26 GMT
Sorry, I forgot to say that the numberOfTaskSlots is always 6..

On Fri, Feb 5, 2016 at 3:32 PM, Flavio Pompermaier <pompermaier@okkam.it>
wrote:

> Hi to all,
>
> I'm testing how to speed up my Flink job and I faced the following
> situations in my *6 nodes* cluster (where each node has 8 CPUs) and 1
> node does also the job manager:
>
> Scenario 1:
>
>    - # of network buffers 4096
>    - parallelism: 36
>    - *The job fails because I have not enough network buffers*
>
> Scenario 2:
>
>    - # of network buffers *8192*
>    - parallelism: 36
>    - *The job ends successfully in about 20 minutes *
>
> Scenario 3:
>
>    - # of network buffers *4096*
>    - 6 nodes
>    - parallelism: *6*
>    - *The job ends successfully in about 11 minutes*
>
> What can I infer from those results? That my job is I/O bounded thus
> having more threads in the same machine accessing simultaneously to the
> disk downgrade the performance of the pipeline?
>
> Best,
> Flavio
>

Mime
View raw message