flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8741) KafkaFetcher09/010/011 uses wrong user code classloader
Date Fri, 23 Feb 2018 11:35:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374241#comment-16374241

ASF GitHub Bot commented on FLINK-8741:

GitHub user tzulitai opened a pull request:


    [FLINK-8741] [kafka] Fix incorrect user code classloader in FlinkKafkaConsumer

    ## What is the purpose of the change
    Fixes the incorrectly used user code classloader in the Flink Kafka Consumer.
    Since 010 / 011 versions reuses the `Kafka09Fetcher`, this fix affects all three versions.
    To guard this for the future, the `Kafka010Example` has been adapted to demonstrate using
a custom watermark assigner. This would have allowed our end-to-end tests to catch the issue.
    ## Brief change log
    The end-to-end tests for Kafka now should be failing, without the user code classloader
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (yes / **no**)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes
/ **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    ## Documentation
      - Does this pull request introduce a new feature? (yes / **no**)
      - If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tzulitai/flink FLINK-8741

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5568
commit 45ff7baa78aa7cb0e798865dddda36e2f0d00087
Author: Tzu-Li (Gordon) Tai <tzulitai@...>
Date:   2018-02-23T11:25:06Z

    [FLINK-8741] [kafka] Fix incorrect user code classloader in FlinkKafkaConsumer
    This commit fixes incorrectly using the parent of the user code class
    loader. Since Kafka 010 / 011 versions directly reuse 09 code, this fix
    fixes the issue for all versions.
    This commit also extends the Kafka010Example, so that is uses a custom
    watermark assigner. This allows our end-to-end tests to have caught this


> KafkaFetcher09/010/011 uses wrong user code classloader
> -------------------------------------------------------
>                 Key: FLINK-8741
>                 URL: https://issues.apache.org/jira/browse/FLINK-8741
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector
>    Affects Versions: 1.5.0, 1.4.1
>            Reporter: Tzu-Li (Gordon) Tai
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Blocker
>             Fix For: 1.5.0, 1.4.2
> This commit https://github.com/apache/flink/commit/0a1ce0060ef3af29b196ab6ad58f97e49a40a4cf#diff-51fb939365cf758a89794a2599344702R98 caused
the wrong classloader to be used.
> The user code classloader should be used directly, not its parent. That change seems
to be irrelevant to the issue, and seems to have been changed by accident.

This message was sent by Atlassian JIRA

View raw message