hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4334) Add support for CPU isolation/monitoring of containers
Date Thu, 02 Aug 2012 15:12:03 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427374#comment-13427374
] 

Arun C Murthy commented on MAPREDUCE-4334:
------------------------------------------

Maybe I haven't been able to communicate this clear enough, please let me try again:

I'd strongly go for a model where platform-specific features (e.g. cgroups, setuid etc.) are
supported via the native code and *build* system (autotool chain) so that we can, from the
end-user perspective, automatically deal with them via a single controlling configuration
knob i.e. yarn.nodemanager.container-executor in this case. 

The alternative, which is various Java interfaces are much worse since now you have to configure
yarn.nodemanager.container-executor, the resource-enforcer etc. This can also have configuration
errors such as TasksetEnforcer in RHEL6 or CgroupsEnforcer in RHEL5 etc.

The native code is, simply, far simpler option which puts the onus on us and takes the burden
away from the end-user or admin.

Thoughts?
                
> Add support for CPU isolation/monitoring of containers
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-4334
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4334
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Andrew Ferguson
>         Attachments: MAPREDUCE-4334-executor-v1.patch, MAPREDUCE-4334-executor-v2.patch,
MAPREDUCE-4334-executor-v3.patch, MAPREDUCE-4334-executor-v4.patch, MAPREDUCE-4334-pre1.patch,
MAPREDUCE-4334-pre2-with_cpu.patch, MAPREDUCE-4334-pre2.patch, MAPREDUCE-4334-pre3-with_cpu.patch,
MAPREDUCE-4334-pre3.patch, MAPREDUCE-4334-v1.patch, MAPREDUCE-4334-v2.patch, mapreduce-4334-design-doc-v2.txt,
mapreduce-4334-design-doc.txt
>
>
> Once we get in MAPREDUCE-4327, it will be important to actually enforce limits on CPU
consumption of containers. 
> Several options spring to mind:
> # taskset (RHEL5+)
> # cgroups (RHEL6+)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message