openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Thömmes <markusthoem...@apache.org>
Subject Re: Prototyping for a future architecture
Date Tue, 28 Aug 2018 12:19:33 GMT
Hi all,

Am Mo., 27. Aug. 2018 um 20:04 Uhr schrieb David P Grove <groved@us.ibm.com
>:

>
>
>
> "Markus Thömmes" <markusthoemmes@apache.org> wrote on 08/23/2018 04:19:33
> PM:
>
> >
> > Key point I want to make is: At some point we'll have to start to
> prototype
> > things out and see if our assumptions actually hold water. For example,
> my
> > assumption on a work-stealing backend is pretty much in the air.
> >
> > My proposal for going forward would be:
> > 1. Create a playground for the implementation of some parts of the system
> > (a new repository?)
> > 2. Let's build some of the things that are uncontroversial and absolutely
> > needed in any case (ContainerRouter, ContainerManager).
> > 3. Play around with them, hook them up in different ways, see what works
> > and what doesn't.
> >
> > Some things will need some testing out to see the scale that the
> components
> > can operate at. These things will narrow or widen the solution space for
> > the more controversial topics around how to distribute containers in the
> > system, how to balance between the routers, work-stealing queue: yes/no
> etc.
> >
> > Having some simple components fleshed out could encourage innovation and
> > creates some facts that we need to focus things into a good direction.
> >
> > What do you think? Too early to start with this and/or the wrong way of
> > doing it?
> >
>
> +1 for starting to prototype.  It's been a good discussion and I think
> we've identified some things that we know we don't know, so time to
> experiment and find out.
>
>
> Not sure what the best logistics are for this.  Would like the work to be
> at Apache (community visibility).  I'm not sure if the best way is a new
> repo or an experimental branch of the main repo (we could dial down the
> level of testing on the branch to make it less cumbersome?).  The branch is
> attractive to me because it might make it easier to keep in synch with the
> components we aren't changing.
>

I actually think we should generate a new repository for this. Opening PRs
etc. will then not clutter the "main" repository and we don't need to worry
about breaking anything.

If nobody objects I'm going to get "incubator-openwhisk-protoype" generated
and will create a rough outline in the repository (different folders for
different parts of the system).

One more basic question (and this is going to be controversial): Most of
the componentry in the execution layer will have to be build anew. I'd like
to switch the implementation language of at least the ContainerRouter to
Golang. Why? Because scale/performance matters a lot for them. As Dave
mentioned on multiple occasions, it will greatly matter how many containers
such a Router can handle under load and that scale will define the
implementation alternatives we will have at hand. I therefore would like to
optimise these Routers for minimal overhead. We could go even lower level,
but I guess that'd be at a kinda big maintenance cost.

I can envision the ContainerManager to be simpler to implement in Golang as
well, at least for some of the deployment alternatives (Kubernetes comes to
mind). The Golang based clients seemed superior to me vs. clients in any
other language.

As all of the communication will probably be HTTP only anyway, these
implementations should be swappable anytime.

Comments very welcome! Let me know your opinions.

Cheers,
Markus

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message