openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From TzuChiao Yeh <su3g4284zo...@gmail.com>
Subject Re: Prototyping for a future architecture
Date Wed, 29 Aug 2018 09:41:16 GMT
Hi Markus,

Thanks! Yes, the design is based on you've put out a few weeks ago. The
biggest difference is there's no ContainerRouter, instead, implement in
LoadBalancer impl. The terminology I've changed for shorter Scala-based
project naming convention, but basically follow up your idea. On the
detailed protocols and design, I've tried my own way, and because there's a
deadline in gsoc schedule, that I choose the way I'm more comfortable and
can only afford subset of overall design. i.e. use akka message passing
instead of http, no shared queues, no kube, reuse invoker-agent for
pause/unpause, no error path, no log and may not race-free, etc. I don't
expect to reuse anything in this implementation, hope this won't lead to
misunderstanding around folks.

Thanks,
Tzu-Chiao

On Wed, Aug 29, 2018 at 5:00 PM Markus Thömmes <markusthoemmes@apache.org>
wrote:

> Hi Tzu-Chiao,
>
> first of all: This looks amazing! As the terminology is similar: Is this a
> separate design or is this based on what I've put out a few weeks ago?
>
> In general: Everybody is eligible to join the prototyping effort, I think
> that's the whole point of having that discussion here. You're very welcome
> to join there and it seems you'll even be more knowledgeable on some of the
> things than anybody else, so it'd be awesome to get your experience in
> there!
>
> Cheers,
> Markus
>
> Am Mi., 29. Aug. 2018 um 10:48 Uhr schrieb TzuChiao Yeh <
> su3g4284zo6y7@gmail.com>:
>
> > Hi,
> >
> > I'm wondering if someone who pay interests aside from core teams (like
> me)
> > are eligible to join prototyping? I know it will increase the maintenance
> > effort on managing all things being visible: opening and labeling issues,
> > plans, assignees, etc. If yes, from my point of view, a standalone repo
> > with issue tracker and pull requests system might be easier to achieve
> the
> > visibility and communication. Will it be possible to use submodule if
> > there's a code-reuse or clarifying migration path concerned?
> >
> > For prototyping:
> > I've drafted a small prototype [1] during last months. Bare me to
> introduce
> > some background about me: some folks might know that, I'm a master
> student
> > and have attended google summer of code this year with openwhisk,
> mentored
> > by Rodric and Carlos. The prototype is the submitted result for final
> work
> > during last few months, although it's buggy, ill-designed and
> not-finished
> > yet. However, if it's ready to start prototyping now, bare me to showoff
> > this under a poor state, as the purpose of my past work, hope this help
> to
> > explore ideas before going to deeper and collaboration. (I've asked most
> of
> > issues I've found in previous threads and may be solved in mind between
> > folks however.)
> >
> > [1] https://tz70s.github.io/posts/openwhisk-performance-improvement/
> >
> > Thanks,
> > Tzu-Chiao
> >
> > On Wed, Aug 29, 2018 at 11:31 AM Markus Thömmes <
> markusthoemmes@apache.org
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > okay, let's separate concerns:
> > >
> > > ### Golang vs. Scala vs. something else entirely: ###
> > >
> > > I'm okay doing it in Scala although I don't see an issue in swapping
> the
> > > language for a component that needs to be rewritten from the ground up
> > > anyway (which I'm suggesting at least the ContainerRouter is).
> > >
> > > The ContainerPool/ContainerProxy part of it comes to mind immediately.
> It
> > > was built with a scale of maybe 100 containers max in mind. There are a
> > few
> > > performance problems with it at scale, that I can point out off the
> bat.
> > > The changes needed to mitigate those are rather severe and while we
> > **can**
> > > implement those (and maybe even should if this proofs to make sense) I
> > > believe it makes sense to at least experiment with a freshly written
> > > component. In the new architecture, we can also greatly reduce the
> state
> > > model to just paused/unpaused for the ContainerRouter, further
> > simplifying
> > > its implementation.
> > >
> > > The ContainerRouter's speed requirements are more close to those of
> > > nginx/envoy than they are to those of the current ContainerPool/Proxy
> > > implementation. As Dave mentioned, for the ContainerRouter it will be
> > very
> > > critical to know how many containers it can handle in reality, hence
> I'm
> > > shooting for an as efficient implementation of its innerts as possible
> to
> > > flesh out how far we can push the boundaries here.
> > >
> > > In any case: Yes we can stick to Scala here, but at least the
> > > ContainerRouter might be 100% new code and we might not even be able to
> > > rely on things like akka-http for its implementation for the reasons
> > > mentioned above.
> > >
> > > ### Seperate repo vs. branch ###
> > >
> > > Towards Michaels points: The prototype repo is supposed to be used for
> > > experimenting/prototyping and for that purpose only. As Dave pointed
> out
> > > above, we'd want these experiments to happen with Apache visibility.
> > There
> > > will never be a completely working system available in that repository.
> > The
> > > goal of it is to find out if certain assumptions of a new design are
> > > feasible or not. The goal is **not** to build productised components
> that
> > > we can take over to the mainline.
> > >
> > > I always thought of that rearch as an incremental step as well and it
> > > shouldn't need major changes to the API (or at least none that we are
> not
> > > already introducing right now). I don't think we're going to see the
> true
> > > potential of this though if we try to take the system's components as
> is
> > > and reuse them in contexts they have never been built for in the first
> > > place.
> > >
> > > As soon as we have clearance over which way we want to ultimately go
> with
> > > the architecture, I agree we should come up with a nice migration plan
> > and
> > > start building it into the main repository through that migration plan.
> > The
> > > prototyping repository should vanish at some point.
> > >
> > > ### Summary ###
> > >
> > > I don't feel super strong about a choice of language, I just thought
> I'd
> > > throw that in the ring as well. As sentiments seem quite strong against
> > > doing that right now, which is perfectly fine, I'd be fine doing the
> > > implementations in Scala. The disclaimer though is, that I'd like to
> > write
> > > critical components afresh to not have to fight with existing
> > architectural
> > > choices to much but to be able to think freely about the problems at
> > hand.
> > >
> > > Which is why I do feel quite strongly about not doing this in a branch.
> > The
> > > structure of the project will be very different and I can easily see
> > people
> > > confused if we start having pull-requests against a "prototype" branch
> > > mixed in with our usual pull-requests.
> > >
> > > I do however agree that we eventually will need to build this in an
> > > incremental way, once we know what we want to build in the first place.
> > For
> > > me, it's off the table to attempt to rewrite the system from scratch,
> > we'll
> > > likely never get that done (I think that's version 2 syndrome
> Michael?).
> > I
> > > agree that might end up in a dead end and we shouldn't do that.
> > >
> > > Does that make the intention more clear? I hope I addressed all the
> > points
> > > asked. Opinions welcome :)
> > >
> > > Cheers,
> > > Markus
> > >
> > > Am Mi., 29. Aug. 2018 um 00:55 Uhr schrieb Dascalita Dragos <
> > > ddragosd@gmail.com>:
> > >
> > > > I also click on the clean and lean upgrade path, with gradual
> > > improvements
> > > > to the existing system (which is production-proof), as Michael and
> > Rodric
> > > > are suggesting.
> > > >
> > > > At the same time I see how Go would make sense from the Kubernetes
> and
> > > > Knative impl POV, how the trends favor Go over Scala [1], [2], and
> the
> > > > potential benefits from addressing a larger dev audience. @Markus,
> feel
> > > > free to keep me honest if this is where you're coming from.
> > > >
> > > > Trying to keep our options open, I'm thinking: why not take Markus'
> > > > challenge, and design these enhancements now to support polyglot
> > > > implementations in the future ? This would be the important decision
> we
> > > can
> > > > take. BTW, for the data plane itself, we could go at even lower
> levels
> > to
> > > > Envoy or Nginx, to provide superior performance, should we choose to.
> > > >
> > > > [1] -
> > > >
> > > >
> > >
> >
> https://trends.google.com/trends/explore?geo=US&q=%2Fm%2F091hdj,%2Fm%2F09gbxjr
> > > >
> > > > [2] - https://octoverse.github.com/
> > > >
> > > > On Tue, Aug 28, 2018 at 2:53 PM Rodric Rabbah <rodric@gmail.com>
> > wrote:
> > > >
> > > > > Thanks Michael for raising these points. I share the same opinion
> and
> > > > > sentiment and think a branch with a clean migration story is better
> > and
> > > > > makes more sense. I am not entirely convinced that the choice of
> > > language
> > > > > itself will make the difference vs the new architecture which is
> > quite
> > > > > different and should in itself be more efficient.
> > > > >
> > > > > -r
> > > > >
> > > > > On Tue, Aug 28, 2018 at 4:51 PM Michael Marth
> > <mmarth@adobe.com.invalid
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Markus,
> > > > > >
> > > > > > IMHO what you propose below is a rather severe change in scope
of
> > > this
> > > > > > discussion and effort.
> > > > > > Up until so far this was about _evolving_ the OW architecture.
We
> > > have
> > > > > not
> > > > > > explicitly discussed it, but one could assume that it is at
least
> > > > > feasible
> > > > > > to gradually adopt the new architecture. So there would be a
> smooth
> > > > path
> > > > > > between the current state of the code base and a future one.
> > > > > >
> > > > > > Your proposal below breaks this assumption somewhat (by
> proposing a
> > > new
> > > > > > repo instead of a branch - which will inevitably make the 2
code
> > > bases
> > > > > > drift apart) as well as explicitly by suggesting a new
> > implementation
> > > > > > language. Especially the latter would create a schism between
> > OW-now
> > > > and
> > > > > > OW-future.
> > > > > > This schism has implications like the perception of OW-now being
> > > > > > deprecated, the _possibility_ of no clean upgrade path, the
> > immediate
> > > > > split
> > > > > > of the community between *-now and *-future and of course carries
> > the
> > > > > risk
> > > > > > of the version 2 syndrome.
> > > > > >
> > > > > > I would propose to implement the future architecture in a branch
> > and
> > > in
> > > > > > Scala first. If it turns out to be good, then subsequent
> > experiments
> > > > can
> > > > > > show or not-show if a switch of language is of additional value.
> > That
> > > > > would
> > > > > > allow to make a decision based on data rather than anything
else.
> > > > > >
> > > > > > My2c
> > > > > > Michael
> > > > > >
> > > > > >
> > > > > > On 28.08.18, 14:26, "Markus Thömmes" <markusthoemmes@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > >     Hi all,
> > > > > >
> > > > > >     Am Mo., 27. Aug. 2018 um 20:04 Uhr schrieb David P Grove
<
> > > > > > groved@us.ibm.com
> > > > > >     >:
> > > > > >
> > > > > >     >
> > > > > >     >
> > > > > >     >
> > > > > >     > "Markus Thömmes" <markusthoemmes@apache.org>
wrote on
> > > 08/23/2018
> > > > > > 04:19:33
> > > > > >     > PM:
> > > > > >     >
> > > > > >     > >
> > > > > >     > > Key point I want to make is: At some point we'll
have to
> > > start
> > > > to
> > > > > >     > prototype
> > > > > >     > > things out and see if our assumptions actually
hold
> water.
> > > For
> > > > > > example,
> > > > > >     > my
> > > > > >     > > assumption on a work-stealing backend is pretty
much in
> the
> > > > air.
> > > > > >     > >
> > > > > >     > > My proposal for going forward would be:
> > > > > >     > > 1. Create a playground for the implementation
of some
> parts
> > > of
> > > > > the
> > > > > > system
> > > > > >     > > (a new repository?)
> > > > > >     > > 2. Let's build some of the things that are
> uncontroversial
> > > and
> > > > > > absolutely
> > > > > >     > > needed in any case (ContainerRouter, ContainerManager).
> > > > > >     > > 3. Play around with them, hook them up in different
ways,
> > see
> > > > > what
> > > > > > works
> > > > > >     > > and what doesn't.
> > > > > >     > >
> > > > > >     > > Some things will need some testing out to see
the scale
> > that
> > > > the
> > > > > >     > components
> > > > > >     > > can operate at. These things will narrow or widen
the
> > > solution
> > > > > > space for
> > > > > >     > > the more controversial topics around how to distribute
> > > > containers
> > > > > > in the
> > > > > >     > > system, how to balance between the routers, work-stealing
> > > > queue:
> > > > > > yes/no
> > > > > >     > etc.
> > > > > >     > >
> > > > > >     > > Having some simple components fleshed out could
encourage
> > > > > > innovation and
> > > > > >     > > creates some facts that we need to focus things
into a
> good
> > > > > > direction.
> > > > > >     > >
> > > > > >     > > What do you think? Too early to start with this
and/or
> the
> > > > wrong
> > > > > > way of
> > > > > >     > > doing it?
> > > > > >     > >
> > > > > >     >
> > > > > >     > +1 for starting to prototype.  It's been a good discussion
> > and
> > > I
> > > > > > think
> > > > > >     > we've identified some things that we know we don't
know, so
> > > time
> > > > to
> > > > > >     > experiment and find out.
> > > > > >     >
> > > > > >     >
> > > > > >     > Not sure what the best logistics are for this.  Would
like
> > the
> > > > work
> > > > > > to be
> > > > > >     > at Apache (community visibility).  I'm not sure if
the best
> > way
> > > > is
> > > > > a
> > > > > > new
> > > > > >     > repo or an experimental branch of the main repo (we
could
> > dial
> > > > down
> > > > > > the
> > > > > >     > level of testing on the branch to make it less
> cumbersome?).
> > > The
> > > > > > branch is
> > > > > >     > attractive to me because it might make it easier to
keep in
> > > synch
> > > > > > with the
> > > > > >     > components we aren't changing.
> > > > > >     >
> > > > > >
> > > > > >     I actually think we should generate a new repository for
> this.
> > > > > Opening
> > > > > > PRs
> > > > > >     etc. will then not clutter the "main" repository and we
don't
> > > need
> > > > to
> > > > > > worry
> > > > > >     about breaking anything.
> > > > > >
> > > > > >     If nobody objects I'm going to get
> > "incubator-openwhisk-protoype"
> > > > > > generated
> > > > > >     and will create a rough outline in the repository (different
> > > > folders
> > > > > > for
> > > > > >     different parts of the system).
> > > > > >
> > > > > >     One more basic question (and this is going to be
> > controversial):
> > > > Most
> > > > > > of
> > > > > >     the componentry in the execution layer will have to be build
> > > anew.
> > > > > I'd
> > > > > > like
> > > > > >     to switch the implementation language of at least the
> > > > ContainerRouter
> > > > > > to
> > > > > >     Golang. Why? Because scale/performance matters a lot for
> them.
> > As
> > > > > Dave
> > > > > >     mentioned on multiple occasions, it will greatly matter
how
> > many
> > > > > > containers
> > > > > >     such a Router can handle under load and that scale will
> define
> > > the
> > > > > >     implementation alternatives we will have at hand. I therefore
> > > would
> > > > > > like to
> > > > > >     optimise these Routers for minimal overhead. We could go
even
> > > lower
> > > > > > level,
> > > > > >     but I guess that'd be at a kinda big maintenance cost.
> > > > > >
> > > > > >     I can envision the ContainerManager to be simpler to
> implement
> > in
> > > > > > Golang as
> > > > > >     well, at least for some of the deployment alternatives
> > > (Kubernetes
> > > > > > comes to
> > > > > >     mind). The Golang based clients seemed superior to me vs.
> > clients
> > > > in
> > > > > > any
> > > > > >     other language.
> > > > > >
> > > > > >     As all of the communication will probably be HTTP only
> anyway,
> > > > these
> > > > > >     implementations should be swappable anytime.
> > > > > >
> > > > > >     Comments very welcome! Let me know your opinions.
> > > > > >
> > > > > >     Cheers,
> > > > > >     Markus
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> > Tzu-Chiao Yeh (@tz70s)
> >
>


-- 
Tzu-Chiao Yeh (@tz70s)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message