openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Santana <>
Subject Re: Mesos integration
Date Wed, 31 May 2017 21:24:45 GMT
Hi Tyson

Thanks for reaching out, having Mesos integration in OpenWhisk is something
I have heard some users asking about and any help to figure out how to best
integrate are very welcome.

As you already aware OpenWhisk Invoker is a critical path component
optimized for performance, to get those container pause and unpause in
single digit milliseconds that's the reason for the recent change from
using docker engine down to runc

Here is my initial notes, take into account I don't have first hand
experience with Mesos, which is why is good to have folks like you join :-)

I think extending in a way that a different mode can be use would be useful
to plug a different orchestration methodology.

For log collection we are looking into sending logs into ELK (Elastic
search) there is some early  investigation going on, so I see after this
integration, mesos can be told to which ELK to send the logs, and the
Invoker to get the results.

In terms of scala vs. nodejs, I would recommend to use scala with the java
client, since there could be some components being integrated into Invoker
and code sharing.

How you want to follow up, maybe you have something to show us or maybe
setup a hangout call to meet

I think Dragos from Adobe, might have a lot more feedback than me since I
think he has first hand experience


On Sat, May 27, 2017 at 2:44 PM Tyson Norris <>

> Hi All -
> I’ve been considering how we can leverage mesos resource scheduling within
> OpenWhisk, and wanted to share some ideas and solicit some feedback.
> In general, I’ve been experimenting with an approach that replaces the
> DockerContainer implementation with a MesosTask. The MesosTask will have
> effectively the same lifecycle (ignoring pause/resume functions) and the
> same interface (HTTP api to the container), the main difference being that
> the container will be deployed to some arbitrary host in the cluster.
> There are a few broad topics around this, including:
> - docker client usage needs to be better isolated (e.g. cleaning up
> existing containers, reconciling containers already running at time of
> invoker startup, etc). I think this will be mostly straightforward.
> - log collection - I’m not sure the best approach here, but one is to
> completely decouple log collection from activation execution, and then
> provide a mesos-specific impl
> - container state tracking + load balancing - obviously there is potential
> for conflict if 2 invokers schedule activations on the same container in
> the cluster (since the state tracking of the container would still be in
> the invoker). This would imply some extension to the ContainerPool as well.
> - mesos framework - we’ve discussed internally a bit, and some preferences
> I have are: leverage the mesos http api (avoid the older jni libs if
> possible) and provide an independent framework application which provides a
> (simpler)  http api that is consumed by the invoker (if mesos integration
> is enabled in the deployment). This way the framework deployment can be
> isolated from controller/invoker deployment, and interaction with docker
> containers is mostly the same (except for logging).
> - There are a few options of implementing mesos http api clients like:
> rxjava client, and nodejs client, but I've not seen any good Scala client
> to date, so we may either provide a scala app using the rxjava client, or
> create.a new client in Scala.
> Let me know if you have any thoughts around any of these, and I will share
> more details as they come.
> Thanks
> Tyson

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message