beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <>
Subject [jira] [Commented] (BEAM-3261) Apex runner does not detect pipeline failure
Date Tue, 28 Nov 2017 23:57:00 GMT


Eugene Kirpichov commented on BEAM-3261:

It looks like a separate issue to me. I filed

> Apex runner does not detect pipeline failure
> --------------------------------------------
>                 Key: BEAM-3261
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-apex
>            Reporter: Eugene Kirpichov
>            Assignee: Thomas Weise
>            Priority: Blocker
> I was looking at and asked myself "Don't
we already have a ValidatesRunner test for this?"
> Turns out we do:
> I ran this test with TestApexRunner, and I observed the same exception as in
, however the test passed.
> It seems that it is passing because ApexRunnerResult.waitUntilFinish() detects only PAssert
assertion errors, but not any other errors:
> This seems very problematic, as it means that 1) the status of Apex ValidatesRunner tests
cannot be trusted: if they fail in any other way other than a PAssert error, the failure will
be undetected, and 2) in general, a user can not trust a successful from Apex
> For tests in particular, some other TestXRunner's guard against such a failure mode by
verifying not only that there were no assertion failures, but also that all assertions succeeded
- using metrics: e.g.
. Fixing this would be optimal, but meanwhile, the runner should at least fail the pipeline
in case of error.

This message was sent by Atlassian JIRA

View raw message