hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2157) tasklauncher threads in TaskTracker can die because of unexpected interrupts
Date Tue, 26 Oct 2010 20:24:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925117#action_12925117
] 

Devaraj Das commented on MAPREDUCE-2157:
----------------------------------------

When the TT re-inits, the launcher threads are interrupted and they are created again in the
initialization (just stating the obvious). Are you sure that there was no issue in the creation
of the launcher threads? I want to see if we can narrow down the problem further before going
down the route of gating the thread exit on the running flag. I am still not convinced that
if the interrupted status was set, it wasn't intentional (intentional -> TaskTracker.close())...

> tasklauncher threads in TaskTracker can die because of unexpected interrupts
> ----------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2157
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2157
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Joydeep Sen Sarma
>            Assignee: Joydeep Sen Sarma
>            Priority: Critical
>
> taskLauncher thread exits on interruptedException and on Interrupt conditions without
checking for any shutdown flag:
>      while (!Thread.interrupted()) {
>         ...
>         } catch (InterruptedException e) { 
>           return; // ALL DONE                                                       
                                                              
>         }
>      }
> If the interrupt happened because of reasons other than TaskTracker.close() - then the
TaskTracker will look functional - but will not be able to schedule tasks anymore. worse -
some tasks (that are in the launch queue) will hang indefinitely un UNASSIGNED state (the
JobTracker will not even time them out). We have seen this cause jobs to hang indefinitely.
> It seems that the interrupted condition can be set by log4j (of which there are many
calls inside TaskLauncher). See or instance: http://logging.apache.org/log4j/1.2/xref/org/apache/log4j/AsyncAppender.html

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message