hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4906) TaskTracker running out of memory after running several tasks
Date Wed, 17 Dec 2008 19:19:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657494#action_12657494
] 

Arun C Murthy commented on HADOOP-4906:
---------------------------------------

A user complained about this on core-user@ supplying the following trace:
{noformat}
java.lang.OutOfMemoryError: Java heap space 
at java.util.Arrays.copyOf(Arrays.java:2882) 
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) 
at java.lang.StringBuffer.append(StringBuffer.java:224) 
at com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.getNodeValueString(DeferredDocumentImpl.java:1167)
at com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.getNodeValueString(DeferredDocumentImpl.java:1120)
at com.sun.org.apache.xerces.internal.dom.DeferredTextImpl.synchronizeData(DeferredTextImpl.java:93)
at com.sun.org.apache.xerces.internal.dom.CharacterDataImpl.getData(CharacterDataImpl.java:160)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:928)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:851)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:819) 
at org.apache.hadoop.conf.Configuration.get(Configuration.java:278) 
at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:446) 
at org.apache.hadoop.mapred.JobConf.getKeepFailedTaskFiles(JobConf.java:308) 
at org.apache.hadoop.mapred.TaskTracker$TaskInProgress.setJobConf(TaskTracker.java:1506)
at org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:727)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:721) 
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1306) 
at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:946) 
at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1343) 
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2354)
{noformat}

> TaskTracker running out of memory after running several tasks
> -------------------------------------------------------------
>
>                 Key: HADOOP-4906
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4906
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Arun C Murthy
>            Priority: Blocker
>             Fix For: 0.20.0
>
>
> Looks like the TaskTracker isn't cleaning up correctly after completed/failed tasks,
I suspect that the JobConfs aren't being deallocated. Eventually the TaskTracker runs out
of memory after running several tasks.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message