Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Mon, 16 Jul 2012 19:00:43 +0000 (UTC)
From: "Robert Joseph Evans (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <93902675.59692.1342465243529.JavaMail.jiratomcat@issues-vm>
In-Reply-To: <1513010632.3133.1339433923183.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Commented] (MAPREDUCE-4334) Add support for CPU
 isolation/monitoring of containers
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415524#comment-13415524 ] 

Robert Joseph Evans commented on MAPREDUCE-4334:
------------------------------------------------

I agree with Bikas and Arun to a point.  I can see some situations, like running a multi-tenent Hadoop cloud where you do want strict isolation.  So that the people who are paying a premium to get consistent results from their part of the cluster never have to worry about someone else doing something really bad on another part of the cluster.  Is this enough of a concern to make it the default, I would say no.  Is it enough of a concern to make it an option that comes with and is maintained by Hadoop, that is TBD, I don't plan on running my clusters that way, but I am not the only Hadoop customer.  Arun, didn't you mention something at Hadoop Summit about some discussions you had with people who want full VMs to run their containers in specifically for isolation purposes?

As for memory spikes, at least on Linux I thought you could configure swap on Linux containers so that if a container goes over its budget, i.e. spikes, then it swaps to disk instead of launching the OOM killer. I could be wrong, I have not dug into it very much.
                
> Add support for CPU isolation/monitoring of containers
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-4334
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4334
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Andrew Ferguson
>         Attachments: MAPREDUCE-4334-pre1.patch, MAPREDUCE-4334-pre2-with_cpu.patch, MAPREDUCE-4334-pre2.patch, MAPREDUCE-4334-pre3-with_cpu.patch, MAPREDUCE-4334-pre3.patch
>
>
> Once we get in MAPREDUCE-4327, it will be important to actually enforce limits on CPU consumption of containers. 
> Several options spring to mind:
> # taskset (RHEL5+)
> # cgroups (RHEL6+)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira