flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7456) Implement Netty sender incoming pipeline for credit-based
Date Wed, 16 Aug 2017 16:28:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129035#comment-16129035

ASF GitHub Bot commented on FLINK-7456:

GitHub user zhijiangW opened a pull request:


    [FLINK-7456][network]Implement Netty sender incoming pipeline for credit-based

    ## What is the purpose of the change
    This PR is based on #4533 whose commits are also included for passing travis. Review the
last commit for this PR change.
    On sender side, it maintains credit from receiver's `PartitionRequest` and `AddCredit`
messages, then sends buffer based on credit and network capacity. This PR is mainly involved
in incoming pipeline logic for credit-based.
    ## Brief change log
      - *Each subpartition view maintains current credit and a boolean field to mark whether
it is already registered available for transfer*
      - *Update current credit in processing `PartitionRequest` and `AddCredit` messages*
      - *The mechanism of enqueue the subpartition view and update the registered status field*
    ## Verifying this change
    This change added tests and can be verified as follows:
      - *Added test to verify that current credit is updated correctly and subpartition view
is enqueued when received `AddCredit` message*
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (no)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (no)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (no)
    ## Documentation
      - Does this pull request introduce a new feature? (no)
      - If yes, how is the feature documented? (not applicable)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zhijiangW/flink FLINK-7456

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4552
commit e35c1ff8066bf44344495d132a1092b9db3ef182
Author: Zhijiang <wangzhijiang999@aliyun.com>
Date:   2017-08-07T09:31:17Z

    [FLINK-7378][core]Create a fix size (non rebalancing) buffer pool type for the floating

commit 969c24d3bf80c1ff89ada11e81b9bf4fea14066f
Author: Zhijiang <wangzhijiang999@aliyun.com>
Date:   2017-08-14T06:30:47Z

    [FLINK-7394][core]Implement basic InputChannel with free buffers,credit and backlog

commit 15fa828449d73f53042c57e9c5494d75ddee575f
Author: Zhijiang <wangzhijiang999@aliyun.com>
Date:   2017-08-10T05:29:13Z

    [FLINK-7406][network]Implement Netty receiver incoming pipeline for credit-based

commit d0674244f15701863a5dd3f68b7274b3bd49c64d
Author: Zhijiang <wangzhijiang999@aliyun.com>
Date:   2017-08-12T14:13:25Z

    [FLINK-7416][network] Implement Netty receiver outgoing pipeline for credit-based

commit 6eaff7877ad43eab674e184153365b50ec8e1559
Author: Zhijiang <wangzhijiang999@aliyun.com>
Date:   2017-08-16T13:24:53Z

    [FLINK-7456][network]Implement Netty sender incoming pipeline for credit-based


> Implement Netty sender incoming pipeline for credit-based
> ---------------------------------------------------------
>                 Key: FLINK-7456
>                 URL: https://issues.apache.org/jira/browse/FLINK-7456
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>            Reporter: zhijiang
>            Assignee: zhijiang
>             Fix For: 1.4.0
> This is a part of work for credit-based network flow control.
> On sender side, each subpartition view maintains an atomic integer {{currentCredit}}
from receiver. Once receiving the messages of {{PartitionRequest}} and {{AddCredit}}, the
{{currentCredit}} is added by deltas.
> Each view also maintains an atomic boolean field to mark it as registered available for
transfer to make sure it is enqueued in handler only once. If the {{currentCredit}} increases
from zero and there are available buffers in the subpartition, the corresponding view will
be enqueued for transferring data.

This message was sent by Atlassian JIRA

View raw message