flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-9142) Lower the minimum number of buffers for incoming channels to 1
Date Mon, 28 May 2018 11:30:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-9142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Aljoscha Krettek updated FLINK-9142:
    Fix Version/s:     (was: 1.5.1)

> Lower the minimum number of buffers for incoming channels to 1
> --------------------------------------------------------------
>                 Key: FLINK-9142
>                 URL: https://issues.apache.org/jira/browse/FLINK-9142
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>    Affects Versions: 1.5.0, 1.6.0
>            Reporter: Nico Kruber
>            Priority: Major
>             Fix For: 1.6.0
> Even if we make the floating buffers optional, we still require {{taskmanager.network.memory.buffers-per-channel}}
number of (exclusive) buffers per incoming channel with credit-based flow control while without,
the minimum was 1 and only the maximum number of buffers was influenced by this parameter.
> {{taskmanager.network.memory.buffers-per-channel}} is set to {{2}} by default with the
argumentation that this way we will have one buffer available for netty to process while a
worker thread is processing/deserializing the other buffer. While this seems reasonable, it
does increase our minimum requirements. Instead, we could probably live with {{1}} exclusive
buffer and up to {{gate.getNumberOfInputChannels() * (networkBuffersPerChannel - 1) + extraNetworkBuffersPerGate}}
floating buffers. That way we will have the same memory footprint as before with only slightly
changed behaviour.

This message was sent by Atlassian JIRA

View raw message