openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Santana <csantan...@gmail.com>
Subject Re: Mesos integration
Date Wed, 31 May 2017 22:19:36 GMT
Yeah I agree between Mesos and Kubernetes work there should be some common
integration extension points that would serve both use cases to some extent



On Wed, May 31, 2017 at 5:42 PM Ben Browning <bbrownin@redhat.com> wrote:

> I believe a lot of Tyson's observations around Mesos integration also
> apply to Kubernetes integration. For example, I've been experimenting
> with an approach that replaces the DockerContainer with a
> KubernetesContainer. The KubernetesContainer also ignores the pause
> and resume of functions and also deploys containers to arbitrary nodes
> in the Kubernetes cluster.
>
> Here's a rough but working (except for log shipping) prototype of that
> -
> https://github.com/projectodd/incubator-openwhisk/blob/kube-container/core/invoker/src/main/scala/whisk/core/containerpool/kubernetes/KubernetesContainer.scala
>
>
> So, I'd like to add a +1 Tyson's notes around isolating docker client
> usage and log collection. I believe they both equally apply to a
> Kubernetes environment as well. And, more broadly, we'll need to
> figure out how to make the various Container implementations pluggable
> (or at least configurable) for Docker, Kubernetes, & Mesos.
>
> Ben
>
>
> On Wed, May 31, 2017 at 5:24 PM, Carlos Santana <csantana23@gmail.com>
> wrote:
> > Hi Tyson
> >
> > Thanks for reaching out, having Mesos integration in OpenWhisk is
> something
> > I have heard some users asking about and any help to figure out how to
> best
> > integrate are very welcome.
> >
> > As you already aware OpenWhisk Invoker is a critical path component
> > optimized for performance, to get those container pause and unpause in
> > single digit milliseconds that's the reason for the recent change from
> > using docker engine down to runc
> >
> > Here is my initial notes, take into account I don't have first hand
> > experience with Mesos, which is why is good to have folks like you join
> :-)
> >
> > I think extending in a way that a different mode can be use would be
> useful
> > to plug a different orchestration methodology.
> >
> > For log collection we are looking into sending logs into ELK (Elastic
> > search) there is some early  investigation going on, so I see after this
> > integration, mesos can be told to which ELK to send the logs, and the
> > Invoker to get the results.
> >
> > In terms of scala vs. nodejs, I would recommend to use scala with the
> java
> > client, since there could be some components being integrated into
> Invoker
> > and code sharing.
> >
> > How you want to follow up, maybe you have something to show us or maybe
> > setup a hangout call to meet
> >
> > I think Dragos from Adobe, might have a lot more feedback than me since I
> > think he has first hand experience
> >
> > --Carlos
> >
> >
> >
> > On Sat, May 27, 2017 at 2:44 PM Tyson Norris <tnorris@adobe.com.invalid>
> > wrote:
> >
> >> Hi All -
> >> I’ve been considering how we can leverage mesos resource scheduling
> within
> >> OpenWhisk, and wanted to share some ideas and solicit some feedback.
> >>
> >> In general, I’ve been experimenting with an approach that replaces the
> >> DockerContainer implementation with a MesosTask. The MesosTask will have
> >> effectively the same lifecycle (ignoring pause/resume functions) and the
> >> same interface (HTTP api to the container), the main difference being
> that
> >> the container will be deployed to some arbitrary host in the cluster.
> >>
> >> There are a few broad topics around this, including:
> >> - docker client usage needs to be better isolated (e.g. cleaning up
> >> existing containers, reconciling containers already running at time of
> >> invoker startup, etc). I think this will be mostly straightforward.
> >> - log collection - I’m not sure the best approach here, but one is to
> >> completely decouple log collection from activation execution, and then
> >> provide a mesos-specific impl
> >> - container state tracking + load balancing - obviously there is
> potential
> >> for conflict if 2 invokers schedule activations on the same container in
> >> the cluster (since the state tracking of the container would still be in
> >> the invoker). This would imply some extension to the ContainerPool as
> well.
> >> - mesos framework - we’ve discussed internally a bit, and some
> preferences
> >> I have are: leverage the mesos http api (avoid the older jni libs if
> >> possible) and provide an independent framework application which
> provides a
> >> (simpler)  http api that is consumed by the invoker (if mesos
> integration
> >> is enabled in the deployment). This way the framework deployment can be
> >> isolated from controller/invoker deployment, and interaction with docker
> >> containers is mostly the same (except for logging).
> >> - There are a few options of implementing mesos http api clients like:
> >> rxjava client, and nodejs client, but I've not seen any good Scala
> client
> >> to date, so we may either provide a scala app using the rxjava client,
> or
> >> create.a new client in Scala.
> >>
> >> Let me know if you have any thoughts around any of these, and I will
> share
> >> more details as they come.
> >>
> >> Thanks
> >> Tyson
> >>
> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message