spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steveloughran <>
Subject [GitHub] spark pull request #17364: [SPARK-20038] [core]: move the currentWriter=null...
Date Mon, 20 Mar 2017 20:05:48 GMT
GitHub user steveloughran opened a pull request:

    [SPARK-20038] [core]: move the currentWriter=null assignments into finally {} …

    ## What changes were proposed in this pull request?
    have the`FileFormatWriter.ExecuteWriteTask.releaseResources()` implementations  set `currentWriter=null`
in a finally clause. This guarantees that if the first call to `currentWriter()` throws an
exception, the second releaseResources() call made during the task cancel process will not
trigger a second attempt to close the stream. 
    ## How was this patch tested?
    Tricky. I've been fixing the underlying cause when I saw the problem (HADOOP-14204), but
SPARK-10109 shows I'm not the first to have seen this. I can't replicate it locally any more,
my code no longer being broken.
    code review, however, should be straightforward

You can merge this pull request into a Git repository by running:

    $ git pull stevel/SPARK-20038-close

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #17364
commit 725740b49a2b37392699092b1b0e08c63a6152ff
Author: Steve Loughran <>
Date:   2017-03-20T19:54:51Z

    SPARK-20038: move the currentWriter=null assignments into finally {} clauses
    Change-Id: I1e07f5b90ba1a2b05978b1d65876d746d07d1f3c


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message