hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-4089) Hung Tasks never time out.
Date Fri, 30 Mar 2012 19:30:28 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robert Joseph Evans updated MAPREDUCE-4089:
-------------------------------------------

    Attachment: MR-4089.txt

This patch addresses the timeout issue, and it does so by making ping not update progress.
 It is still not completely compatible with 1.0, as in 1.0 if the timeout is set to 0 the
task will never timeout.  But because this patch makes it so ping is ignored a task that has
a timeout of 0, but is so locked up that it cannot ping anymore will never timeout.

I am planning to address these in a follow on JIRA, unless someone has some objections to
doing so.

I also have not run all of the unit tests yet.
                
> Hung Tasks never time out. 
> ---------------------------
>
>                 Key: MAPREDUCE-4089
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4089
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.2, 2.0.0, trunk
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Blocker
>         Attachments: MR-4089.txt
>
>
> The AM will timeout a task through mapreduce.task.timeout only when it does not hear
from the task within the given timeframe.  On 1.0 a task must be making progress, either by
reading input from HDFS, writing output to HDFS, writing to a log, or calling a special method
to inform it that it is still making progress.
> This is because on 0.23 a status update which happens every 3 seconds is counted as progress.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message