beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <>
Subject [jira] [Created] (BEAM-1712) doesn't actually waitUntilFinish
Date Tue, 14 Mar 2017 00:30:41 GMT
Eugene Kirpichov created BEAM-1712:

             Summary: doesn't actually waitUntilFinish
                 Key: BEAM-1712
             Project: Beam
          Issue Type: Bug
          Components: runner-flink, sdk-java-core, testing
            Reporter: Eugene Kirpichov
            Assignee: Stas Levin
            Priority: Blocker

it calls waitUntilFinish() only if
1) run wasn't called
2) enableAutoRunIfMissing is true.

However in practice both of these are false.
1) run() is, in most tests, called. So effectively if you call .run() at all, then this thing
doesn't call waitUntilFinish().
2) enableAutoRunIfMissing() is set to true only via TestPipeline.enableAutoRunIfMissing(),
which is called only from its own unit test.

This means that, for all tests that use TestPipeline - if the test waits until finish, it's
only because of the grace of the particular runner. Which is like really bad.

We're lucky because in practice TestDataflowRunner, TestApexRunner, TestSparkRunner in run()
call themselves waitUntilFinish().

However, TestFlinkRunner doesn't - i.e. there currently might be tests that actually fail
in Flink runner, undetected.

The proper fix to this is to fix TestPipeline to always waitUntilFinish().
Currently testing a quick-fix in to make sure Flink
is safe.

This message was sent by Atlassian JIRA

View raw message