flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8759) Bump Netty to 4.0.56
Date Fri, 23 Feb 2018 15:19:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374480#comment-16374480

ASF GitHub Bot commented on FLINK-8759:

GitHub user NicoK opened a pull request:


    [FLINK-8759][network] preparations for the update of netty to version 4.0.56

    ## What is the purpose of the change
    Based on the changes from #5570, this PR prepares a netty version bump by circumventing
a bug we had with the change from Netty 4.0.27 to 4.0.28 with the old non-credit based flow
control paths:
    Versions >= 4.0.28 contain an improvement by Netty, which slices a Netty buffer instead
of doing a memory copy (https://github.com/netty/netty/issues/3704) in the `LengthFieldBasedFrameDecoder`.
In some situations, this interacts badly with our Netty pipeline leading to `OutOfMemory`
    With credit-based flow control this problem should not exist anymore which is why we can
use the original netty code there.
    ## Brief change log
    - override `LengthFieldBasedFrameDecoder#extractFrame()` and implement two different code
paths depending on whether credit based flow control is on or not
    ## Verifying this change
    This change is a trivial rework / code cleanup without any test coverage.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): **no**
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
      - The serializers: **no**
      - The runtime per-record code paths (performance sensitive): **no** (only per buffer)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: **no**
      - The S3 file system connector: **no**
    ## Documentation
      - Does this pull request introduce a new feature? **no**
      - If yes, how is the feature documented? **JavaDocs**

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/NicoK/flink flink-8759

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5571
commit 272899e01c2836a2a5b47958db7d9d9f6cbf471d
Author: Nico Kruber <nico@...>
Date:   2018-02-23T12:56:29Z

    [FLINK-8768][network] Let NettyMessageDecoder inherit from LengthFieldBasedFrameDecoder
    This replaces one additional step from the pipeline and does not only remove
    overhead there but also allows use to override the #extractFrame() method to
    restore the old Netty 4.0.27 behaviour for non-credit based code paths which
    had a bug with Netty >= 4.0.28 there (see FLINK-8759).

commit 11f673b415bce310cc2195fb6778051c095083fa
Author: Nico Kruber <nico@...>
Date:   2018-02-23T13:06:00Z

    [FLINK-8759][network] preparations for the update of netty to version 4.0.56


> Bump Netty to 4.0.56
> --------------------
>                 Key: FLINK-8759
>                 URL: https://issues.apache.org/jira/browse/FLINK-8759
>             Project: Flink
>          Issue Type: Improvement
>          Components: Network
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Major
>             Fix For: 1.5.0
> For a bug in Netty's shutdown sequence and overall improvements in Netty, I'd like to
bump the version (and stay within the 4.0 series for now). The problem we faced in the past
should not be relevant for credit-based flow control anymore and can be worked around (for
the fallback code path) by restoring {{LengthFieldBasedFrameDecoder}}'s old behaviour of copying
contents to new buffers instead of slicing the existing one (please refer to FLINK-7428 for
the inverse direction).

This message was sent by Atlassian JIRA

View raw message