flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8747) The tag of waiting for floating buffers in RemoteInputChannel should be updated properly
Date Thu, 22 Feb 2018 14:49:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372887#comment-16372887
] 

ASF GitHub Bot commented on FLINK-8747:
---------------------------------------

GitHub user zhijiangW opened a pull request:

    https://github.com/apache/flink/pull/5558

    [FLINK-8747][bugfix] The tag of waiting for floating buffers in RemoteInputChannel should
be updated properly

    ## What is the purpose of the change
    
    *In credit-based flow control mode, when the number of available buffers is less than
required buffers (backlog + initialCredit), the `RemoteInputChannel` will request floating
buffers from `BufferProvider`. If not get enough available floating buffers, the `RemoteInputChannel`
registers itself as listener in `BufferProvider` and updates the tag `isWaitingForFloatingBuffers`
as true to avoid registration repeatedly.*
    
     *When a floating buffer is recycled to `BufferProvider`, it will notify the listener
of available buffer. But the listener may not need floating buffers currently if the available
buffers is not less than required buffers, then the floating buffers will be returned to BufferProvider
directly. Most importantly, the tag `isWaitingForFloatingBuffers` should also be updated as
false, otherwise the `RemoteInputChannel` will not request floating buffers any more after
the available buffers less than required buffers.*
    
    *There are two scenarios for causing the above issue:*
    
    - The recycled exclusive buffers increase the total available buffers which is equal to
or more than required buffers.
    - The decreased sender's backlog resulting the available buffers equal to required buffers.
    
    ## Brief change log
    
      - *Updates the tag `isWaitingForFloatingBuffers` false on `notifyBufferAvailable` if
current available buffers is not less than required buffers*
    
    ## Verifying this change
    
    This change added tests and can be verified as follows:
    
      - *Modify `RemoteInputChannelTest#testAvailableBuffersLessThanRequiredBuffers to cover
this case*
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (no)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (no)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (no)
      - The S3 file system connector: (no)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (no)
      - If yes, how is the feature documented? (not applicable)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zhijiangW/flink FLINK-8747

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5558.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5558
    
----
commit 10ad6db14d072a5208c45c4be08624ebd7e8ea13
Author: Zhijiang <wangzhijiang999@...>
Date:   2018-02-22T14:41:38Z

    [FLINK-8747][bugfix] The tag of waiting for floating buffers in RemoteInputChannel should
be updated properly

----


> The tag of waiting for floating buffers in RemoteInputChannel should be updated properly
> ----------------------------------------------------------------------------------------
>
>                 Key: FLINK-8747
>                 URL: https://issues.apache.org/jira/browse/FLINK-8747
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>    Affects Versions: 1.5.0
>            Reporter: zhijiang
>            Assignee: zhijiang
>            Priority: Minor
>
> In credit-based flow control mode, when the number of available buffers is less than
required buffers (backlog + initialCredit), the {{RemoteInputChannel}} will request floating
buffers from {{BufferProvider}}. If not get enough available floating buffers, the {{RemoteInputChannel}}
registers itself as listener in {{BufferProvider}} and updates the tag {{isWaitingForFloatingBuffers}}
as {{true}} to avoid registration repeatedly.
>  
> When a floating buffer is recycled to {{BufferProvider}}, it will notify the listener
of available buffer. But the listener may not need floating buffers currently if the available
buffers is not less than required buffers, then the floating buffers will be returned to {{BufferProvider}}
directly. Most importantly, the tag {{isWaitingForFloatingBuffers}} should also be updated as {{false,
otherwise the RemoteInputChannel}} will not request floating buffers any more after the
available buffers less than required buffers.
>  
> There are two scenarios for causing the above issue:
>  * The recycled exclusive buffers increase the total available buffers which is equal
to or more than required buffers.
>  * The decreased sender's backlog resulting the available buffers equal to required buffers.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message