hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tarun Parimi (Jira)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-7278) Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false
Date Thu, 07 May 2020 15:44:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101800#comment-17101800
] 

Tarun Parimi commented on MAPREDUCE-7278:
-----------------------------------------

Adding new patch to fix unit test failure. Not sure how to add new unit test to cover this
since this patch ensures the original behavior with speculation disabled is achieved

> Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative
are false
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-7278
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7278
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 2.8.0
>            Reporter: Tarun Parimi
>            Priority: Major
>         Attachments: MAPREDUCE-7278.001.patch, MAPREDUCE-7278.002.patch, Screen Shot
2020-04-30 at 8.04.27 PM.png
>
>
> When a failed task attempt container is stuck in FAIL_FINISHING_CONTAINER state for some
time, we observe two task attempts are launched simultaneously even when speculative execution
is disabled.
> This results in the below message shown in the killed attempts, indicating speculation
has occurred. This is an issue for jobs which require speculative execution to be strictly
disabled.
>   !Screen Shot 2020-04-30 at 8.04.27 PM.png!
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message