hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-7121) Add Abortable support to mapreduce tasks
Date Tue, 10 Jul 2018 21:25:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539251#comment-16539251
] 

ASF GitHub Bot commented on MAPREDUCE-7121:
-------------------------------------------

GitHub user bschell opened a pull request:

    https://github.com/apache/hadoop/pull/402

    MAPREDUCE-7121. add Abortable so that we can abort output

    This patch adds abortable to mapreduce outputs
    
    In the case of using a remote filesystem with mapreduce, upon task attempt failure we
ensure the file output stream gets aborted before being closed which prevents partial results
from being uploaded.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/bschell/hadoop bschelle/abortable

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/hadoop/pull/402.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #402
    
----
commit 1d0c3932299f0bc9a8a92b41048626ab953df533
Author: Scheller <bschelle@...>
Date:   2018-07-04T00:13:32Z

    add Abortable so that we can abort output
    
    This adds abortable so we can abort output in the case of a task attempt that is successful
even though it failed to generate output files

----


> Add Abortable support to mapreduce tasks
> ----------------------------------------
>
>                 Key: MAPREDUCE-7121
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7121
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: client
>    Affects Versions: 3.0.2
>            Reporter: Brandon Scheller
>            Priority: Major
>             Fix For: 3.2.0
>
>
> This patch adds abortable to mapreduce outputs
> In the case of using a remote filesystem with mapreduce, upon task attempt failure we
ensure the file output stream gets aborted before being closed which prevents partial results
from being uploaded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message