hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-834) When TaskTracker config use old memory management values its memory monitoring is diabled.
Date Wed, 19 Aug 2009 06:28:15 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744915#action_12744915
] 

Hemanth Yamijala commented on MAPREDUCE-834:
--------------------------------------------

Few comments:

- Memory allotted for slot based on old configuration should not be based on getMaxVirtualMemoryForTask(),
but on JobConf.MAPRED_TASK_DEFAULT_MAXVMEM_PROPERTY. Also note that this value will be in
bytes, while the system maintains everything else in MB. So, it should be converted to MB.
- testTaskMemoryMonitoringWithDeprecatedConfiguration should also set the TT configuration
for JobConf.MAPRED_TASK_DEFAULT_MAXVMEM_PROPERTY in bytes instead of MAPRED_TASK_MAXVMEM_PROPERTY.

> When TaskTracker config use old memory management values its memory monitoring is diabled.
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-834
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-834
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Karam Singh
>         Attachments: mapreduce-834-1.patch
>
>
> TaskTracker memory config values -:
> mapred.tasktracker.vmem.reserved=8589934592
> mapred.task.default.maxvmem=2147483648
> mapred.task.limit.maxvmem=4294967296
> mapred.tasktracker.pmem.reserved=2147483648
> TaskTracker start as -:
>                2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The
variable mapred.tasktracker.vmem.reserved is no longer used
> 		2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.tasktracker.pmem.reserved
is no longer used
> 		2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.default.maxvmem
is no longer used
> 		2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.limit.maxvmem
is no longer used
> 		2009-08-05 12:39:03,308 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread:
Map-events fetcher for all reduce tasks on <tracker_name>
> 		2009-08-05 12:39:03,309 INFO org.apache.hadoop.mapred.TaskTracker:  Using MemoryCalculatorPlugin
: org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@19be4777
> 		2009-08-05 12:39:03,311 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks
is -1. TaskMemoryManager is disabled.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message