hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (Naga)" <garlanaganarasi...@huawei.com>
Subject RE: CPU isolation with cgroups and RHEL 6
Date Tue, 17 Nov 2015 19:42:59 GMT
Hi David,

We too have faced this many times and as alternative where OS cannot be upgraded we have CPU
isolation based on CPUset which is not same as existing which based on period & quota.
It doesnt guarantee the usage among containers is balanced efficiently but atleast will ensure
the cpu usage of the yarn containers in a node doesnt exceed the number of cores assigned
for it.




From: David Villegas [dville00@gmail.com]
Sent: Tuesday, November 17, 2015 21:42
To: user@hadoop.apache.org
Subject: CPU isolation with cgroups and RHEL 6

Hi all,

We are running some benchmarks (in the order of 1TB of data) while enabling CPU isolation
using cgroups in YARN containers. However, we are often hitting kernel panics in RHEL 6 with
kernel 2.6.32-504.el6.x86_64. We've tried using RHEL 7 and so far we haven't hit the same

Has anyone else seen this? I found some people are experiencing similar problems (YARN-4048),
so I wonder if there's any kind of consensus about ditching RHEL / CentOS 6 for this type
of setup and just supporting v7, or if it is possible to make it work with some tweaking.

David Villegas.

View raw message