flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8306) FlinkKafkaConsumerBaseTest has invalid mocks on final methods
Date Wed, 10 Jan 2018 07:28:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319816#comment-16319816
] 

ASF GitHub Bot commented on FLINK-8306:
---------------------------------------

Github user tzulitai commented on the issue:

    https://github.com/apache/flink/pull/5200
  
    Regarding the race condition you mentioned:
    hmm, I can't seem to exactly nail down the "and only first `successfulCommits.inc()` can
be omitted because of that" case you mentioned, could you elaborate on that a bit more?
    
    But yes, it seems like there certainly is a race condition here, and can even cause an
NPE on:
    https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/KafkaConsumerThread.java#L496
    
    Seems like the `nextOffsetsToCommit` and its callback instance should be bundled as a
single `AtomicReference` here ...
    
    Would you like to open a PR for that? I wanted to ask because you discovered it in the
first place; if not I'll open a fix for it.


> FlinkKafkaConsumerBaseTest has invalid mocks on final methods
> -------------------------------------------------------------
>
>                 Key: FLINK-8306
>                 URL: https://issues.apache.org/jira/browse/FLINK-8306
>             Project: Flink
>          Issue Type: Bug
>          Components: Kafka Connector, Tests
>            Reporter: Tzu-Li (Gordon) Tai
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Critical
>             Fix For: 1.5.0, 1.4.1
>
>
> The {{FlinkKafkaConsumerBaseTest}} has invalid mocks on a final {{AbstractFetcher::commitInternalOffsetsToKafka(...)}}
method. While an easy fix would be to simply make that method non-final, that is not ideal
since it would be best that the method is left final to prevent overrides in subclasses.
> This suggests that offset committing functionality is too tightly coupled with the {{AbstractFetcher}},
making it hard to perform concise tests to verify offset committing.
> I suggest that we decouple record fetching and offset committing as separate services
behind different interfaces. We should introduce a new interface, say {{KafkaOffsetCommitter}},
and test against that instead. Initially, we can simply let {{AbstractFetcher}} implement
{{KafkaOffsetCommitter}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message