openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Allen <>
Subject Re: Moving out runtime images and tests
Date Wed, 13 Sep 2017 06:57:53 GMT

Seems like its inevitable as 50 min build times are really long!

I think all runtimes should be split out. The unit tests for OpenWhisk itself shouldn't need
a runtime anyway. Integration tests for core could pick a runtime at random to pull in.

> On 12 Sep 2017, at 21:02, Markus Thömmes <> wrote:
> Sounds all sensible to me.
> To 1: Sounds like the right thing to do, if versions are common enough? Like major bumps
could probably use different repos but I guess we can just try and find out what works. As
a starter, the list you posted sounds sensible to me!

I agree with Tyson that one repo per runtime-type is a good idea. If we're going to release
multiple major language versions simultaneously, then we need separate repositories at the
version level  (e.g. openwhisk-action-nodejs6, openwhisk-action-nodejs8, openwhisk-action-php7,
etc) or we need to use branches within a runtime-type repo and have a release system that
knows which branches are "release" ones. I suspect separate repositories is easier, but it's
"just" tooling.

> 2. By Integration Tests I meant: Do the repos need to standup a whole OpenWhisk system
and then run integrations tests or can we get away with just unit tests. I think, the best
would be the latter since its the most agile but we might not be quite there yet. Again something
we should just try and will discover on the way I think!

For all the split out runtimes, I think they obviously need unit tests, but we are going to
need need integration tests for each one as it'll be really easy to miss a change to either
what is sent to the runtime or what is expected to be sent back from it. It'll be really easy
to end up with something broken if we're not careful.

Another possibility is that we have a separate set of unit tests that all the runtimes use.
Detected a change in runtime-unit-tests and getting travis to run the unit tests for all the
runtime repositories could be a challenge though.

> 3. Yep, it's probably just build and push the images on each commit just like we do for
> One more thing (hehe): Do we support independent tags of runtime images vs. the deployed
system? Today we use the same tag that we use to deploy the system to pull the runtime images.
There might be something left to do here like: Do we need a specific version for each runtime
(most flexible) or can we get away with just using latest for each runtime (easiest to maintain).
Versioning might be needed to be able to make breaking changes but still pin versions and
to keep production systems stable that way (otherwise they might pull an incompatible image
while in action). Thinking about it: We need independent versioning per runtime.

Each runtime needs its own version I think. I think that python2 and python3 are going to
need their own separate version, so as noted above, separate repositories for each major language
version would simplify life. 

I think we'll need to pin major version numbers of the runtimes to OpenWhisk for release.
e.g. it'll be a pain if the latest version of OpenWhisk 1.x currently works with openwhisk-action-nodejs6:1.4.1,
then when we release openwhisk-action-nodejs6:1.5.0, the next release of OpenWhisk 1.x should
just pick it up without having to modify an commit a file change in OpenWhisk core. However
OpenWhisk 1.x should not pick up openwhisk-action-nodejs6:2.0.0.

Essentially, I think that we're going to have to start committing to a stable API between
core and the runtimes if we everything's all separate.

Finally, for what it's worth, I'd quite like to see the CLI also split out and again, integration
tests to ensure it works with OpenWhisk core.



> Am 12. September 2017 um 21:56 schrieb Rodric Rabbah <>:
>> +1 good points Tyson.
>> -r
>>> On Sep 12, 2017, at 1:25 PM, Tyson Norris <> wrote:
>>> +1 in general
>>>> Tasks we'd need to do:
>>>> 1. Setup a repo per runtime.
>>> Maybe repo per runtime-type, instead of literally each runtime - specifically
names like: openwhisk-action-nodejs, openwhisk-action-java, openwhisk-action-dockerskeleton,
openwhisk-action-php, openwhisk-action-python, openwhisk-action-swift
>>> (Since each type should share much of the same tests, I think.)
>>>> 2. Move the runtime build + tests there (testwise I would rather copy and
own some dependencies than trying to go DRY. The current setup for dependent repos needs quite
some cleanup and is super hard to maintain for updates in the main repo). We can discuss if
we need Integration Tests for each of the runtimes or if the "unit" tests we have are sufficient
>>> We can DRY it *later* if wanted by releasing mvn artifacts of test base classes
(or all of tests) from core, or something like that, but until that is implemented, agree
that copying is better.
>>> By “Integration Tests” do you mean to run as part of core tests? I think
something minimal would be good like “test that at least nodejs runtime works” (e.g. health
check), but not “test every runtime as part of core integration tests"
>>>> 3. Implement a release process for the runtime images to Dockerhub.
>>> This should be the same as the release process for core images right?
>>> Thanks
>>> Tyson
>>>> The runtimes update fairly rarely so I wouldn't really bother with too strict
of a versioning there, at least not for the first shot. Process wise it does seem straightforwardly
>>>> What do you think?
>>>> Cheers
>>>> Markus

Development thoughts at
Daily Jotter for macOS at

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message