hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Ferguson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4334) Add support for CPU isolation/monitoring of containers
Date Tue, 12 Jun 2012 17:08:43 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293777#comment-13293777
] 

Andrew Ferguson commented on MAPREDUCE-4334:
--------------------------------------------

hi Arun,

I've actually been looking into this recently myself, and would be happy to take the lead
on it. So far, I've been focusing on cgroups as they also provide memory containment, and
provide a path for managing future resources as well. although taskset is available on RHEL5,
it's not capable of isolating fractions of a CPU.

while cgroups' memory support gives an upper-bound on the amount of memory tasks can consume,
the RHEL6 cpu support is actually a lower-bound. until CFS bandwidth control [1] is more widespread,
we can place tasks judiciously to create guarantees, building on cgroups to ensure the lower-bounds.


best,
Andrew


[1] for a quick overview: http://lwn.net/Articles/428230/ ... more in-depth discussion here:
http://www.kernel.org/doc/ols/2010/ols2010-pages-245-254.pdf
                
> Add support for CPU isolation/monitoring of containers
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-4334
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4334
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>
> Once we get in MAPREDUCE-4327, it will be important to actually enforce limits on CPU
consumption of containers. 
> Several options spring to mind:
> # taskset (RHEL5+)
> # cgroups (RHEL6+)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message