mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Glikson" <GLIK...@il.ibm.com>
Subject Re: custom(ized) conteinerization
Date Sat, 10 Oct 2015 07:31:53 GMT
Timothy Chen <tnachen@gmail.com> wrote on 10/10/2015 09:38:38 AM:
> 
> As Ben said containierizers don't conflict since the framework 
> specifies which containerizer the task should be launched from.
> 
> And isolators are not configurable per task but per slave, so all 
> tasks will use the same isolators in the same slave.

Tasks of the same framework -- sure. But what about tasks/containers of 
different frameworks running on the same host? For example, containerizer 
A might assume that all the network traffic should go via bridge X, while 
containerizer B might assume OVS+vxlan. In such a case the two are likely 
to be incompatible (maybe even cluster-wide, but most certainly within a 
host). I assume that conflicts are also possible with other resources 
(e.g., incompatible ways of using cgroups).

Regards,
Alex

> 
> Tim
> 
> > On Oct 9, 2015, at 11:22 PM, Alex Glikson <GLIKSON@il.ibm.com> wrote:
> > 
> > Thanks!
> > 
> >> You can mix containerizers, although they should not conflict with 
each
> >> other.
> > 
> > How would I know whether they conflict? For example, the docker one 
and 
> > the default mesos one with certain configuration of isolators etc?
> > 
> > Thanks,
> > Alex
> > 
> > 
> > Benjamin Mahler <benjamin.mahler@gmail.com> wrote on 10/10/2015 
03:51:56 
> > AM:
> > 
> >> From: Benjamin Mahler <benjamin.mahler@gmail.com>
> >> To: dev <dev@mesos.apache.org>
> >> Date: 10/10/2015 03:52 AM
> >> Subject: Re: custom(ized) conteinerization
> >> 
> >> (1) is something that has come up before. The containerizer deals 
with a
> >> variable sized container, the semantics of a task are defined by the
> >> executor, there is no way for the containerizer to understand the
> > meaning
> >> of a task currently. Some tasks are "special" in that they don't have 
an
> >> executor (e.g. command task, docker task, etc), and in this case they
> > will
> >> be isolated individually. The main approach that has been discussed 
to
> > my
> >> knowledge is to have the executor leverage the mesos containerizer to
> >> create nested containers:
> > https://issues.apache.org/jira/browse/MESOS-3333
> >> 
> >> For (2) you should implement a custom network _isolator_ that the 
mesos
> >> containerizer can use.
> >> 
> >> With respect to (3), for regular resources the policy used in Mesos 
is
> > that
> >> Mesos will never make decisions to kill things, it must be triggered 
by
> > the
> >> framework or an operator. So this requirement should be met already. 
For
> >> revocable resources, Mesos may destroy containers, but the framework
> > will
> >> be aware of that when deciding to use them. If they are stateful, you
> >> should use reservations and store the data in a volume.
> >> 
> >> You can mix containerizers, although they should not conflict with 
each
> >> other.
> >> 
> >>> On Fri, Oct 9, 2015 at 3:04 AM, Alex Glikson <GLIKSON@il.ibm.com>

wrote:
> >>> 
> >>> Triggered by the thread on potential deprecation of external
> >>> containerizer, I wonder what would make sense to do to address the
> >>> following set of requirements:
> >>> 1. I need resource isolation for individual tasks (mainly for QoS
> >>> reasons), so having container per task seems reasonable
> >>> 2. I have rather advanced networking requirements, not easily
> > addressable
> >>> with default mesos containerizer or docker
> >>> 3. Some of the tasks are stateful, so I would really prefer that 
Mesos
> >>> doesn't kill them, pretty much ever (unless triggered by the
> > framework)
> >>> 
> >>> It seems that having my own containerizer would be a reasonable
> > approach.
> >>> But given some of the requirements above, I am trying to figure out
> >>> whether I would at all need to implement "usage" and "update" (and
> > maybe
> >>> even 'destroy', unless it is invoked as part of killTask received 
from
> > the
> >>> framework?).
> >>> 
> >>> Moreover, the isolation mechanism I have in mind does use the same
> > Linux
> >>> features as docker/mesos containerizers (cgroups, namespaces, etc),
> > but in
> >>> a somewhat different manner. So, I wonder whether I can use more 
than
> > one
> >>> containerizer on the same host -- e.g., to run tasks of my framework
> > on
> >>> the same host as tasks of , say, Marathon+docker (and if yes, how 
can
> > I
> >>> check whether they will work together). If mixing containerizers in
> > the
> >>> same host is not recommended, is there an easy way to dynamically
> > decide
> >>> which slaves are 'allocated' to which 'type' of resources (e.g., 
some
> > sort
> >>> of entire-host allocation policy)?
> >>> 
> >>> Some thoughts/advice would be really helpful, before we actually 
spend
> >>> time implementing a new containerizer, one way or another.
> >>> 
> >>> Thanks!
> >>> Alex
> >>> 
> >>> P.S. Disclaimer: I am new to Mesos, so maybe some (or all) of the
> > above
> >>> doesn't make much sense, so bear with me..
> > 
> > 
> 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message