Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <CAKBpmQqq2HPBvvzL047onPfpFgamHapGowYXubt6DtenYY=6JA@mail.gmail.com>
References: <CAKBpmQqeABTH622wVpWg5h4NOx+z0ortNqg7kT6ko_=jFf__gA@mail.gmail.com>
 <CAGr9p8Be9=ehYRZsuTkRPMyZOxe1w_jMjCRFxJHLpk3y8QDZaw@mail.gmail.com>
 <CAKBpmQo1hE-NLoY2FqG7mJV3qDyEREe5TnNfvrYikRYuxtYDAQ@mail.gmail.com>
 <CAGr9p8AFwAQqk8=2DN3+NVad=sr+XZGyEMuwp5NiG86TiCSORQ@mail.gmail.com>
 <CANMXwW3uQD3jYA3F9TYDkyK=0S04JHq-10_jM_A_SOTqkX9fhQ@mail.gmail.com> <CAKBpmQqq2HPBvvzL047onPfpFgamHapGowYXubt6DtenYY=6JA@mail.gmail.com>
From: Ufuk Celebi <uce@apache.org>
Date: Mon, 9 May 2016 10:37:21 +0200
Message-ID: <CAKiyyaE7dejrB_QMSNLkYUTPW3dcjdGeTeXFiCPY8_ywoS81CA@mail.gmail.com>
Subject: Re: How to choose the 'parallelism.default' value
To: user@flink.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
archived-at: Mon, 09 May 2016 08:38:03 -0000

Hey Punit,

you need to give the task managers more network buffers as Robert
suggested. Using the formula from the docs, can you please use 147456
(96^2*4*4) for the number of network buffers. Each buffer is 32 KB,
meaning that you give 4,5 GB of memory to the network stack. You might
have to adjust the heap memory (taskmanager.heap.mb) you give to the
task managers accordingly.

Does this solve it?

=E2=80=93 Ufuk


On Sat, May 7, 2016 at 10:50 AM, Punit Naik <naik.punit44@gmail.com> wrote:
> I am afraid not.
>
> On 07-May-2016 1:24 PM, "Aljoscha Krettek" <aljoscha@apache.org> wrote:
>>
>> Could it be that the TaskManagers are configured with not-enough memory?
>>
>> On Thu, 5 May 2016 at 13:35 Robert Metzger <rmetzger@apache.org> wrote:
>>>
>>> The default value of taskmanager.network.numberOfBuffers is 2048. I wou=
ld
>>> recommend to use a multiple of that value, for example 16384 (given tha=
t you
>>> have enough memory per TaskManager)
>>>
>>> I recommend checking out these slides I created a while ago. They expla=
in
>>> what the network buffers are needed for:
>>> http://www.slideshare.net/robertmetzger1/apache-flink-hands-on#37
>>>
>>>
>>> On Thu, May 5, 2016 at 1:30 PM, Punit Naik <naik.punit44@gmail.com>
>>> wrote:
>>>>
>>>> Yes I followed it and changed it to 298 but again it said the same
>>>> thing. The only change was that it now said "required 298, but only 20=
0
>>>> available".
>>>>
>>>> Why did it say that?
>>>>
>>>> On Thu, May 5, 2016 at 4:50 PM, Robert Metzger <rmetzger@apache.org>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I think you've chosen a good initial value for the parallelism.
>>>>> The higher the parallelism, the more network buffers are needed. I
>>>>> would follow the recommendation from the exception and increase the n=
umber
>>>>> of network buffers.
>>>>>
>>>>> On Thu, May 5, 2016 at 11:23 AM, Punit Naik <naik.punit44@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Hello
>>>>>>
>>>>>> I was running a program with 'parallelism.default' of 384 as I read =
in
>>>>>> the documentation on Flink's official page that 'parallelism.default=
' is
>>>>>> "the total number of CPUs in the cluster". I have four machines with=
 96
>>>>>> cores on each of them. So 96*4=3D384. But the program thew an error =
saying:
>>>>>>
>>>>>> Caused by: java.io.IOException: Insufficient number of network
>>>>>> buffers: required 384, but only 298 available. The total number of n=
etwork
>>>>>> buffers is currently set to 2048. You can increase this number by se=
tting
>>>>>> the configuration key 'taskmanager.network.numberOfBuffers'.
>>>>>>
>>>>>> What does this mean? And how to choose a proper value for parallelis=
m?
>>>>>>
>>>>>> --
>>>>>> Thank You
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Punit Naik
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Thank You
>>>>
>>>> Regards
>>>>
>>>> Punit Naik