beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <>
Subject [jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner
Date Sat, 05 Aug 2017 00:34:00 GMT


Eugene Kirpichov commented on BEAM-2671:

20820fa5477ffcdd4a9ef2e9340353ed3c5691a9, mentioned above, is the culprit.

At 20820fa5, I ran the test 10 times and it failed 10/10.
At the previous commit b3099bba, it succeeds 10/10.

However, I did git cherry-pick --reverse of that commit on top of current master branch, and
the test still fails - so reverting the commit is not sufficient.

Moreover, according to my understanding of its enclosing PR,
reverting it is really not an option, because before that PR the Spark runner was completely
unusable in streaming mode (not advancing input watermark). Unfortunately the PR does not
seem to include any tests for the fix, but I presume it was tested manually, or perhaps gets
some testing coverage from ValidatesRunner tests.

I propose to postpone the current JIRA to 2.2.0, because:
- Spark Runner currently passes all ValidatesRunner tests (this is not a ValidatesRunner test),
so I presume that it was completely broken, we'd have other evidence.
- Release 2.0.0 does not contain which means that
in that release Spark runner *is* unusable in streaming mode, i.e. even if the current JIRA
signals that it's still broken in some way, it can't be more broken than 2.0.0 - i.e. there
is no regression, and there is most likely some improvement (albeit incomplete).

> CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner
> --------------------------------------------------------------------------------
>                 Key: BEAM-2671
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-spark
>            Reporter: Etienne Chauchot
>            Assignee: Jean-Baptiste Onofré
>             Fix For: 2.1.0, 2.2.0
> Error message:
> Flatten.Iterables/FlattenIterables/FlatMap/ParMultiDo(Anonymous).out0: 
> Expected: iterable over [] in any order
>      but: Not matched: "late"

This message was sent by Atlassian JIRA

View raw message