hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4334) Add support for CPU isolation/monitoring of containers
Date Mon, 16 Jul 2012 19:00:43 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415524#comment-13415524
] 

Robert Joseph Evans commented on MAPREDUCE-4334:
------------------------------------------------

I agree with Bikas and Arun to a point.  I can see some situations, like running a multi-tenent
Hadoop cloud where you do want strict isolation.  So that the people who are paying a premium
to get consistent results from their part of the cluster never have to worry about someone
else doing something really bad on another part of the cluster.  Is this enough of a concern
to make it the default, I would say no.  Is it enough of a concern to make it an option that
comes with and is maintained by Hadoop, that is TBD, I don't plan on running my clusters that
way, but I am not the only Hadoop customer.  Arun, didn't you mention something at Hadoop
Summit about some discussions you had with people who want full VMs to run their containers
in specifically for isolation purposes?

As for memory spikes, at least on Linux I thought you could configure swap on Linux containers
so that if a container goes over its budget, i.e. spikes, then it swaps to disk instead of
launching the OOM killer. I could be wrong, I have not dug into it very much.
                
> Add support for CPU isolation/monitoring of containers
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-4334
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4334
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Andrew Ferguson
>         Attachments: MAPREDUCE-4334-pre1.patch, MAPREDUCE-4334-pre2-with_cpu.patch, MAPREDUCE-4334-pre2.patch,
MAPREDUCE-4334-pre3-with_cpu.patch, MAPREDUCE-4334-pre3.patch
>
>
> Once we get in MAPREDUCE-4327, it will be important to actually enforce limits on CPU
consumption of containers. 
> Several options spring to mind:
> # taskset (RHEL5+)
> # cgroups (RHEL6+)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message