openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Sicker <boa...@gmail.com>
Subject Re: A direction to take in a new component, "the scheduler"
Date Thu, 18 Jul 2019 18:36:06 GMT
This is one of the situations where having some sort of proposal
process can help align the various people and technical considerations
a bit easier than mailing lists and PRs. There are numerous other
communities with similar proposal processes (e.g., SIPs for Scala,
KIPs for Kafka, JEPs and JSRs for Java, etc.). Using such a process
can help obtain active buy-in to these changes more than a PR or dev@
post might depending on the scale of the change.

On Thu, 18 Jul 2019 at 12:49, Sven Lange-Last
<sven.lange-last@de.ibm.com> wrote:
>
> Hello Dominic and other discussion participants,
>
> we are in the great situation that Dominic (style95) volunteers to create
> and verify an alternative OpenWhisk architecture that may solve some of
> the problems that adopters of OpenWhisk may observe when running
> production systems with high utilization. I really appreciate that Dominic
> already invested and is willing to invest so much time.
>
> While I encourage Dominic to experiment with a different architecture, my
> point of view is that it's in the interest of the project as well as all
> adopters running OpenWhisk-based production systems to keep the project
> stable. We need an approach where we can continue to evolve the existing
> architecture and provide bug fixes without being slowed down or affected
> by the work on or problems with the new architecture. At the same time, we
> need a playground where we can work on the new architecture effectively
> without negatively affecting the existing architecture.
>
> For me, this means:
>
> * We need feature flags / switches as suggested by Dominic and others that
> keep the existing architecture enabled by default.
>
> * The standard deployment only configures and installs what is needed for
> the existing architecture. Installing extra components like etcd takes
> more time so that development, build and test cycles take longer. In
> addition, extra components consume more resources so that brittle system
> tests for the existing architecture may fail more often.
>
> * We need a good separation between tests for existing and new
> architecture. System tests are often brittle and need special resources
> like etcd. Such tests for the new architecture must not be part of the
> existing gradle suites. I think that stable, fast-running unit tests can
> be part of existing suites.
>
> * The Travis checks for pull requests must not contain system tests or
> other brittle / long-running tests of the new architecture. We already
> have some brittle tests today that occasionally fail Travis. As Dominic
> suggested, extra build / deployment / test jobs that run separately would
> be a great idea.
>
> I would like to add a different aspect...
>
> My experience is that the observed problems of an OpenWhisk based system
> really depend on utilization and the workload mix (memory limits, action
> duration, action container image, number of distinct actions, ...). And I
> would expect that people running a few activations will have totally
> different observations than those people running a system with hundreds of
> invokers and millions of activations with a diverse workload mix. And
> based on the observed problems, different contributors will probably try
> to solve different problems and set different priorities.
>
> I had a great discussion with Dominic on these aspects. It turned out that
> he observes different problems than I do when running a large production
> environment. I'm very interested in Dominic's proposal - at the same time,
> my impression is that it will not necessarily address the problems I see.
> Still, we can learn a lot from Dominic's work.
>
> At some point in time, the OpenWhisk community probably needs to decide
> whether to keep the existing architecture and evolve it - or to replace it
> with the suggested architecture if it proves useful. My impression is that
> Dominic only received limited feedback on his proposal and I'm guilty of
> providing my feedback much too late. We should discuss the problems we
> observe with running OpenWhisk more openly so that contributors can align
> their work with the need of others.
>
>
> Mit freundlichen Grüßen / Regards,
>
> Sven Lange-Last
> Senior Software Engineer
> IBM Cloud Functions
> Apache OpenWhisk
>
>
> E-mail: sven.lange-last@de.ibm.com
> Find me on:
>
>
> Schoenaicher Str. 220
> Boeblingen, 71032
> Germany
>
>
>
>
> IBM Deutschland Research & Development GmbH
> Vorsitzende des Aufsichtsrats: Martina Koederitz
> Geschäftsführung: Dirk Wittkopp
> Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart,
> HRB 243294
>
>


-- 
Matt Sicker <boards@gmail.com>

Mime
View raw message