hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4906) TaskTracker running out of memory after running several tasks
Date Mon, 05 Jan 2009 11:27:46 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sharad Agarwal updated HADOOP-4906:
-----------------------------------

    Attachment: 4906_v3.patch

Setting the JobConf references to null in TaskInProgress#cleanup had problem since the TaskRunner
thread may need the ref later. Had an offline discussion with Devaraj and decided that TaskRunner#run
would be the better place to clean the references.
This patch makes the JobConf references to null in the finally block of TaskRunner#run. Also
had to keep some data in member variables of MapOutputFile and TaskInProgress so that JobConf
reference is not required when task finishes.

> TaskTracker running out of memory after running several tasks
> -------------------------------------------------------------
>
>                 Key: HADOOP-4906
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4906
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Arun C Murthy
>            Assignee: Sharad Agarwal
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 4906_v1.patch, 4906_v2.patch, 4906_v3.patch
>
>
> Looks like the TaskTracker isn't cleaning up correctly after completed/failed tasks,
I suspect that the JobConfs aren't being deallocated. Eventually the TaskTracker runs out
of memory after running several tasks.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message