Mailing-List: contact dev-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@aurora.apache.org
MIME-Version: 1.0
Date: Fri, 14 Aug 2015 11:39:17 -0700
Message-ID: 
 <CAAATh-aXM8OL+ERwK4mA4_pv+rhMz=ERDutK1XJLcu2GhxpD5g@mail.gmail.com>
Subject: Improvements to container support with custom executor
From: Kevin Sweeney <ksweeney@twitter.com.INVALID>
To: "user@mesos.apache.org" <user@mesos.apache.org>
Cc: dev@aurora.apache.org
Content-Type: multipart/alternative; boundary=001a113a8068f78f43051d49c4e3

--001a113a8068f78f43051d49c4e3
Content-Type: text/plain; charset=UTF-8

(cross-posting for a wider audience)

Hi folks,

With mesos-0.23.0 it looks like a new dependency made it in for TLS
support. While this is fine in theory it actually makes the Docker
Containerizer story very difficult to reason about.

Here's the situation: Aurora uses a custom Python executor for its tasks.
Because of the design of the containerizer the executor runs in the context
of the container. This means that the container has to be able to run the
executor (in practice this means it needs a python2.7 installation and some
shared libraries libmesos links to). Since most containers don't contain
Aurora's executor, Aurora hacked around this by using the executor in
$MESOS_SANDBOX plus a requirement that hosted containers be able to run the
executor. However, with the upgrade to mesos-0.23.0 the containers that
could run the 0.22.0 executor no longer work due to the new dependency on
libcurl-nss.

This is not a problem limited to Docker - I don't see how this design will
work with *any* container runtime - we can never upgrade the executor
without upgrading all the containers to contain its new dependencies, which
at a minimum means we must rebuild them whenever mesos gains a new
dependency.

Does anyone with more experience with these APIs have a suggestion here? It
seems we need to make the executor run in the context of the host OS and
aware of the container (or maybe we have mesos launch a container with the
executor+its dependencies and have it launch a child container).
-- 
Kevin Sweeney
@kts

--001a113a8068f78f43051d49c4e3--