flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed Nader <ahmednader...@gmail.com>
Subject Re: degree of Parallelism
Date Fri, 18 Mar 2016 12:57:30 GMT
Thanks so much Till and Fabian. So if i were to set it, how can i know the
best degree to be used for my application, is it the number of cores or
what should i set it to?

On 18 March 2016 at 13:36, Fabian Hueske <fhueske@gmail.com> wrote:

> Hi,
> did find the documentation for configuring the parallelism [1]?
> It explains how to set the parallelism on different levels: Cluster, Job,
> Task.
> Best, Fabian
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/common/index.html#parallel-execution
> 2016-03-18 13:34 GMT+01:00 Till Rohrmann <trohrmann@apache.org>:
>> Hi Ahmed,
>> if you don't set the parallelism in your program then depending on how
>> you execute your program different parallelisms will be used. If you
>> execute it in your IDE, then the number of cores will be used as
>> parallelism. If you submit it to a cluster without specifying the
>> parallelism via the cli, then the default parallelism which you've
>> specified in flink-conf.yaml will be used. If you use the -p option of the
>> cli, then this value will be used as the parallelism.
>> Cheers,
>> Till
>> On Fri, Mar 18, 2016 at 1:13 PM, Ahmed Nader <ahmednader839@gmail.com>
>> wrote:
>>> Hi,
>>> How can i setParallelism in a generic way that provide better
>>> performance on any device not only mine? In this case is it better to be
>>> set to a certain value or if i just didn't set it to any value does flink
>>> take care of that generically and provide better execution performance?
>>> Thanks.

View raw message