hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-180) task tracker times out cleaning big job
Date Fri, 05 May 2006 18:19:28 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-180?page=all ]

Owen O'Malley updated HADOOP-180:
---------------------------------

    Attachment: task-cleanup-thread.patch

This patch fixes the timeouts by creating a synchronized queue (we really should go to java
1.5 soon) of tasks that need to be cleaned up and a daemon thread that does it in the background.

It also fixes some race conditions in the TaskTracker on the tasks variable. (Some references
where locking the TaskTracker and others were locking the TaskTracker.TaskInProgress.)

I also changed the rpc logging a little to include both client and server time measurements.

> task tracker times out cleaning big job
> ---------------------------------------
>
>          Key: HADOOP-180
>          URL: http://issues.apache.org/jira/browse/HADOOP-180
>      Project: Hadoop
>         Type: Bug

>   Components: mapred
>     Versions: 0.1.1
>     Reporter: Owen O'Malley
>     Assignee: Owen O'Malley
>      Fix For: 0.3
>  Attachments: task-cleanup-thread.patch
>
> After completing a big job (63,920 maps, 1880 reduces, 188 nodes), lots of the TaskTrackers
timed out because the task cleanup is handled by the same thread as the heartbeats.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message