hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1181) Enforce RSS memory limit in TaskMemoryManagerThread
Date Fri, 06 Nov 2009 11:27:32 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774272#action_12774272
] 

Vinod K V commented on MAPREDUCE-1181:
--------------------------------------

bq. This new feature allow us to avoid page swapping which is prone to error.
Can you elaborate on this? RSS unlike vmem is a very dynamic entity for a process, and depends
not just on this process but others too. So I am not sure if trying to shoot down tasks based
on their memory usage will work well.

bq. This new feature allow us to avoid page swapping which is prone to error.
Explain this too?

The original intention why the feature of killing tasks via the TaskMemoryManager was added
was to prevent nodes from going down. If tasks use too much virtual memory (rss *AND* swap),
OS will not have any way of recovering itself. And we have seen instances of this where nodes
go down completely because of this.

On the other hand, I am not too sure too much rss usage results in similar effects. Did you
see such drastic instances? If not and if you are concerned about thrashing only, then a better
way of controlling this may be to not even schedule tasks if total rss usage is to the brim.
Thoughts?


> Enforce RSS memory limit in TaskMemoryManagerThread
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-1181
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1181
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: tasktracker
>    Affects Versions: 0.20.1
>            Reporter: Scott Chen
>             Fix For: 0.20.1
>
>         Attachments: MAPREDUCE-1181.patch
>
>
> TaskMemoryManagerThread will periodically check the rss memory usage of every task. If
the memory usage exceeds the specified threshold, the task will be killed. Also if the total
rss memory of all tasks exceeds (total amount of memory - specified reserved memory). The
task with least progress will be killed to recover the reserved rss memory.
> This is similar to the virtual memory limit provided by TaskMemoryManagerThread. But
now the limit is for rss memory. This new feature allow us to avoid page swapping which is
prone to error.
> The following are the related configurations
> mapreduce.reduce.memory.rss.mb   // RSS memory allowed for a reduce task
> mapreduce.map.memory.rss.mb       // RSS memory allowed for a map task
> mapreduce.tasktracker.reserved.memory.rss.mb     // RSS memory reserved (not for tasks)
on a tasktracker

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message