spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jiangxb1987 <>
Subject [GitHub] spark pull request #20993: [SPARK-23881][CORE][TEST] Fix flaky test JobCance...
Date Fri, 06 Apr 2018 07:42:13 GMT
GitHub user jiangxb1987 opened a pull request:

    [SPARK-23881][CORE][TEST] Fix flaky test JobCancellationSuite."interruptible iterator
of shuffle reader"

    ## What changes were proposed in this pull request?
    The test case JobCancellationSuite."interruptible iterator of shuffle reader" has been
flaky because `KillTask` event is handled asynchronously, so it can happen that the semaphore
is released but the task is still running.
    Actually we only have to check the total number of processed elements if less than the
input elements number, so we know the task get cancelled.
    ## How was this patch tested?
    The new test case still fails without the purposed patch, and succeeded in current master.

You can merge this pull request into a Git repository by running:

    $ git pull JobCancellationSuite

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #20993
commit 965a80f2d02aacff68bae096483c2809e1c9d67e
Author: Xingbo Jiang <xingbo.jiang@...>
Date:   2018-04-06T07:27:23Z

    fix flaky test



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message