spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tgravescs <>
Subject [GitHub] spark pull request: [SPARK-13054] Always post TaskEnd event for ta...
Date Wed, 09 Mar 2016 21:51:09 GMT
Github user tgravescs commented on the pull request:
    To add some more detail,  the executorLost function only calls taskEnded  (which is task
failed) if the task was in the list of successful tasks.  handleSuccessfulTask calls the taskEnded
and then adds it to the list of successful tasks. since taskEnded ends up sending the CompletionEvent
it is possible for the events to show up in either order.  
    But the DAGScheduler.handleTaskCompletion before this PR handled both of those already
and sent SparkListenerTaskEnd in both cases. This PR doesn't change that behavior for those
2 events.
    The messages are success and resubmitted (which is failed).  Both of which DAGScheduler.handleTaskCompletion
would send the SparkListenerTaskEnd event for both with this PR and before this PR.
    If there is anything else you think I missed let me know.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message