hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4906) TaskTracker running out of memory after running several tasks
Date Mon, 05 Jan 2009 11:27:46 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sharad Agarwal updated HADOOP-4906:

    Attachment: 4906_v3.patch

Setting the JobConf references to null in TaskInProgress#cleanup had problem since the TaskRunner
thread may need the ref later. Had an offline discussion with Devaraj and decided that TaskRunner#run
would be the better place to clean the references.
This patch makes the JobConf references to null in the finally block of TaskRunner#run. Also
had to keep some data in member variables of MapOutputFile and TaskInProgress so that JobConf
reference is not required when task finishes.

> TaskTracker running out of memory after running several tasks
> -------------------------------------------------------------
>                 Key: HADOOP-4906
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4906
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Arun C Murthy
>            Assignee: Sharad Agarwal
>            Priority: Blocker
>             Fix For: 0.20.0
>         Attachments: 4906_v1.patch, 4906_v2.patch, 4906_v3.patch
> Looks like the TaskTracker isn't cleaning up correctly after completed/failed tasks,
I suspect that the JobConfs aren't being deallocated. Eventually the TaskTracker runs out
of memory after running several tasks.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message