mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Landells <...@box.com>
Subject help with cgroups isolator on Scientific Linux
Date Fri, 28 Mar 2014 18:05:27 GMT
Hi,

I'm trying to get Mesos up as a PoC at work to see if it's something we
could use effectively.

I'm running SL 6.4, installed libcgroup and ran cgconfig which creates the
cgroup mounts under /cgroup:

[tlandells@tlandells-mesos02.dev ~]$ cat /proc/mounts | grep cgroup
> cgroup /cgroup/cpuset cgroup rw,relatime,cpuset 0 0
> cgroup /cgroup/cpu cgroup rw,relatime,cpu 0 0
> cgroup /cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
> cgroup /cgroup/memory cgroup rw,relatime,memory 0 0
> cgroup /cgroup/devices cgroup rw,relatime,devices 0 0
> cgroup /cgroup/freezer cgroup rw,relatime,freezer 0 0
> cgroup /cgroup/net_cls cgroup rw,relatime,net_cls 0 0
> cgroup /cgroup/blkio cgroup rw,relatime,blkio 0 0


I'm using the mesos_0.17.0_x86_64.rpm package downloaded from
mesosphere.ioand I can't start a slave fine if I use process
isolation, but if I try
cgroups I get the following in /var/log/messages:

2014-03-28T10:46:05.194958-07:00 tlandells-mesos02 mesos-slave[6242]: I0328
> 10:46:05.194540  6242 main.cpp:118] Build: 2014-02-27 15:07:00 by root
> 2014-03-28T10:46:05.194976-07:00 tlandells-mesos02 mesos-slave[6242]:
> I0328 10:46:05.194810  6242 main.cpp:120] Creating "cgroups" isolator
> 2014-03-28T10:46:05.195267-07:00 tlandells-mesos02 mesos-slave[6242]:
> I0328 10:46:05.195050  6242 main.cpp:132] Starting Mesos slave
> 2014-03-28T10:46:05.195657-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@712: Client
> environment:zookeeper.version=zookeeper C client 3.4.5
> 2014-03-28T10:46:05.195667-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@716: Client
> environment:host.name=tlandells-mesos02.dev.box.net.
> 2014-03-28T10:46:05.195671-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@723: Client
> environment:os.name=Linux
> 2014-03-28T10:46:05.195674-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@724: Client
> environment:os.arch=2.6.32-358.18.1.el6.x86_64
> 2014-03-28T10:46:05.195678-07:00 tlandells-mesos02 mesos-slave[6242]:
> I0328 10:46:05.195552  6242 slave.cpp:111] Slave started on 1)@
> 10.3.18.184:5051
> 2014-03-28T10:46:05.195681-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@725: Client
> environment:os.version=#1 SMP Tue Aug 27 14:23:09 CDT 2013
> 2014-03-28T10:46:05.195773-07:00 tlandells-mesos02 mesos-slave[6242]:
> I0328 10:46:05.195708  6242 slave.cpp:211] Slave resources: cpus(*):2;
> mem(*):2808; disk(*):43161; ports(*):[31000-32000]
> 2014-03-28T10:46:05.196105-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@733: Client
> environment:user.name=tlandells
> 2014-03-28T10:46:05.196115-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@741: Client
> environment:user.home=/root
> 2014-03-28T10:46:05.196119-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@log_env@753: Client
> environment:user.dir=/home/tlandells
> 2014-03-28T10:46:05.196122-07:00 tlandells-mesos02 mesos-slave[6242]:
> 2014-03-28 10:46:05,195:6242(0x7fc5fafcb700):ZOO_INFO@zookeeper_init@786:
> Initiating client connection, host=tlandells-mesos01.dev.box.net:2181,
> tlandells-mesos02.dev.box.net:2181,tlandells-mesos03.dev.box.net:2181sessionTimeout=10000
watcher=0x7fc602fc8cb0 sessionId=0
> sessionPasswd=<null> context=0x7fc5d8003b90 flags=0
> 2014-03-28T10:46:05.197040-07:00 tlandells-mesos02 mesos-slave[6242]:
> I0328 10:46:05.196420  6269 cgroups_isolator.cpp:223] Using /cgroup as
> cgroups hierarchy root
> 2014-03-28T10:46:05.197058-07:00 tlandells-mesos02 mesos-slave[6242]:
> Failed to mount cgroups hierarchy at '/cgroup' because we could not remove
> existing directory: Directory not empty


The command being invoked by /usr/bin/mesos-init-wrapper is:

> /usr/local/sbin/mesos-slave --master=zk://
> tlandells-mesos01.dev.box.net:2181,tlandells-mesos02.dev.box.net:2181,
> tlandells-mesos03.dev.box.net:2181/mesos --log_dir=/var/log/mesos
> --isolation=cgroups


I haven't worked with cgroups before so I'm not clear if it's okay to have
Mesos using the default mounts or whether I should have something special
setup. I mean it's clear that Mesos wouldn't be able to empty out /cgroup
because it contains all the cgroup mounts...

I saw a similar thread in the archives from about 6 months ago where the
issue was apparently a clash between Mesos and Docker, but I'm only trying
to run Mesos.

The only thing that might be funky is that these are VMware instances I'm
currently using for testing.

Does anyone have any experience running the cgroups isolator in SL (or even
RH, since it's basically just a rebadge) who can tell me what I'm doing
wrong?

Thanks,
Tony

Mime
View raw message