flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2032) Migrate integration tests from temp output files to collect()
Date Fri, 12 Feb 2016 11:28:18 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15144476#comment-15144476

Fabian Hueske commented on FLINK-2032:

Yes, this issue is still valid. There are still a few tests writing to files, reading the
result and comparing it to the expected result.
I think the best way to find them is to search for usages of {{ExecutionEnvironment.execute()}}
or {{ExecutionEnvironment.execute(String)}}.

In cases where no output is expected (or not validated) it is OK to use the {{DiscardingOutputFormat}}.

Unless, the test tests the output format, tests should be migrated to use {{collect()}}.

> Migrate integration tests from temp output files to collect()
> -------------------------------------------------------------
>                 Key: FLINK-2032
>                 URL: https://issues.apache.org/jira/browse/FLINK-2032
>             Project: Flink
>          Issue Type: Task
>          Components: Tests
>    Affects Versions: 0.9
>            Reporter: Fabian Hueske
>            Priority: Minor
>              Labels: starter
> Most of Flink's integration tests that execute full Flink programs and check their results
are implemented by writing results to temporary output file and comparing the content of the
file to a provided set of expected Strings. Flink's test utils make this quite comfortable
and hide a lot of the complexity of this approach. Nonetheless, this approach has a few drawbacks:
> - increased latency by going through disk
> - comparison is on String representation of objects
> - depends on the file system
> Since Flink's {{collect()}} feature was added, the temp file approach is not the best
approach anymore. Instead, tests can collect the result of a Flink program directly as objects
and compare these against a set of expected objects.
> It would be good to migrate the existing test base to use {{collect()}} instead of temporary
output files.

This message was sent by Atlassian JIRA

View raw message