openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Santana <csantan...@gmail.com>
Subject Re: Updating Package and Language versions for a kind
Date Thu, 27 Jul 2017 14:20:36 GMT
Being thinking more about this, and having a core set of packages included
by default is very useful and I think critical for open source integration
and continues integration in the open.
Also keeping large actions away from the system for folks that just want to
get started and kick the tires using the embedded packages

I think we could have a mid way alternative:

What about if we create new kinds every time we want to pickup an update
for new language update but at the minor (y) level (version x.y.z).
Also include a very small set of core packages, for example request for
nodejs very stable and popular package

We create a new kind to pick up a new minor update and at that time will
pickup the latest version of the packages.
For patch updates, at least for security updates this are done at the patch
level, will update the current kind to update the patch level of the
language.

So I think this aligns with what Rob mentioned about trying to keep kinds
versions at the minor level.

Taking nodejs as an example:
we have nodejs:6 (this is set to 6.9)
let's create a new kind for node nodejs:6.11
This will include latest version of 6.11 and latest versions of packages at
the time of creating of the new kind nodejs:6.11
If there are nodejs patch updates to pickup for 6.11 we update without
creating a new kind will be kept at nodejs:6.11 with the packages pinned to
the versions that they shipped.
I think nodejs moves a bit faster than the other languages so we could
pickup  a new minor nodejs:6.12 or 6.14 when ever the community thinks
there is a minor update from nodejs worth the update to create a new kind.

There might be a gap, I think it's OK for those corner cases that an user
wants to use the latest nodejs version but use old version of the packages,
or use the latest version of the package but an old version of nodejs. I
think for this cases user would need to pinned and include the exact
version of the modules to include and that's OK, I think this will be a
normal thing for people going into production and pinning and controlling
dependencies.

I think it's hard to come up with silver bullet for all languages, as every
language updates at a different speed, and dependencies are more slowest
pace than others.







On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <csantana23@gmail.com> wrote:

> One thought I have being debating is that the open source project doesn't
> include any packages going forward.
> The project will have an image that is small and common denominator that
> the operators deploying decide which packages to add on top.
>
> For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017),
>
> Will keep the version of language nodejs 8 up to date to the latest as
> official releases come out from nodejs
> Operators like IBM and Adobe, RH, others, can base their custom images for
> their offerings on the same base image, adding packages that they want base
> on their offering.
>
> Same for swift new kind swift:4, and so on.
>
> This will be a general policy, as every language has their peculiarity and
> probably we need our runtimes SMEs to talk about the risk on updating or
> not updating per language.
>
> dealing with user level packages is always a pickle for example today we
> have nodejs:6 and we have a set of users that benefit from updating the
> packages to the latest latest, and could have users that they depend on a
> current version of the package and updating the package on them could cause
> problems. If I had my way I would say thought luck for using the embedded
> package and not packaging or doing a bundle build, but serverless is on
> baby years of adoption and I don't want to be thought yet :-)
>
> So for nodejs8 we say thought luck no embedded packages, but will do
> something to make it super easy will have the system help you build/compile
> your action and include your dependencies, these system actions that allow
> users to provide only their code/git-url, and package.json/package.swift,
> and these actions will compile/npm install etc..  and install your action
> including the declared dependencies.
>
>
>
>
>
> On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <mmarth@adobe.com.invalid>
> wrote:
>
>> Hi Rob,
>>
>> Re “I wonder how many deployments roll their own action containers?”
>>
>> I spoke too general (implying what most OW deployments would want). What
>> I can say is that our team wishes to be in control of the libraries that go
>> into the action containers, that’s why we need to roll our own.
>>
>> Agree with the approach you have outlined on patch and minor releases.
>>
>> The topic of deprecation IMHO is the same discussion as when to upgrade
>> language runtimes in action containers: at least for us the deprecation
>> policy is something that we need to be in control of (which is a given as
>> we roll our own containers).
>> +1 to the deprecation tooling you hint at regarding not creating actions
>> for deprecated containers (this is needed regardless of any deprecation
>> policies). This is not covered by the issues you linked to, is it?
>>
>> Cheers
>> Michael
>>
>>
>>
>> On 14/07/17 19:08, "Rob Allen" <rob@akrabat.com> wrote:
>>
>> >Hi Michael, all,
>> >
>> >I wonder how many deployments roll their own action containers? I
>> suspect not many.
>> >
>> >I agree with you both that it would be nice to keep packages (modules)
>> up to date on a regular basis within a kind makes the most sense and a
>> regular schedule would be nice, especially if there's a documented way to
>> do the update for each kind.
>> >
>> >I think we should also keep the language of the kind up to date at the
>> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 -
>> unless there are language kinds that break semver at the patch level? (In
>> which case my head may explode!)
>> >
>> >For minor releases, it's a little more wooly at the moment at least. For
>> NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to 6.11.0
>> without creating a new new "kind". However, for Swift, [PR] [2120] gave us
>> a new `swift:3.1.1` kind rather than updating the `swift3` kind which runs
>> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, so
>> maybe it should have been called `swift:3.1`?
>> >
>> >My thoughts on this is that we should create new kinds for minor
>> language releases on the principle of least surprise for the container
>> users.
>> >
>> >I would like to fairly aggressively deprecate old ones to stop new
>> actions being created for out-of-date language versions, but as deprecating
>> a kind prevents invoking any actions for that kind that's not possible
>> unless some work is done to allow invoking actions on a deprecated kind,
>> but not creating or updating them would be necessary for that to be a
>> possibility.
>> >
>> >I think GitHub Issue [319] [3] is relevant in terms of the tooling
>> around this.
>> >
>> >
>> >Regards,
>> >
>> >Rob...
>> >
>> >[1]: https://github.com/apache/incubator-openwhisk/pull/2078
>> >[2]: https://github.com/apache/incubator-openwhisk/pull/2120
>> >[3]: https://github.com/apache/incubator-openwhisk/issues/319
>> >
>> >
>> >> On 14 Jul 2017, at 16:27, Michael Marth <mmarth@adobe.com.INVALID>
>> wrote:
>> >>
>> >> Rob, all,
>> >>
>> >> I largely echo what James wrote below (except moving this discussion
>> off-list :) )
>> >>
>> >> Incidentally, we had a very similar discussion within my team recently
>> on how to handle this topic for our OW deployment. The bottom line is that
>> the trade-off between stability and being up-to-date can only be decided in
>> a given business context of the OW deployment.
>> >> We (as a project) do not have that context and cannot really make the
>> decision, IMHO. In that light the only reasonable policy in my view is to
>> strive to keep everything updated as best as we can, like monthly or every
>> second month, etc.
>> >> If we miss a beat (i.e. don’t immediately update to latest release) I
>> don’t think that’d be critical. Like James said, most OW providers will
>> have their own runtimes anyway and would (should) take the latest if they
>> need it.
>> >>
>> >> Cheers
>> >> Michael
>> >>
>> >>
>> >> On 13/07/17 11:02, "James Thomas" <jthomas.uk@gmail.com> wrote:
>> >>
>> >>> Good ideas Rob. I had a similar issue when looking at the Swift
>> runtime
>> >>> recently.
>> >>>
>> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
>> >>>
>> >>> Would it be possible to have a scheduled upgrade process for installed
>> >>> modules? Once every four, six or eight weeks? If OpenWhisk did start
>> to
>> >>> produce "releases", it could tie in with that.
>> >>>
>> >>> I'd guess that most people using the built-in packages are more
>> kicking the
>> >>> tires than building production apps. Once you start being a
>> production app,
>> >>> you'll want to explicitly bundle and control your app dependencies.
>> I'd +1
>> >>> on being more aggressive with upgrading module versions.
>> >>>
>> >>> I'd like to have a Github issue to follow for this, I find it easier
>> than
>> >>> the mailing list.
>> >>>
>> >>> On 13 July 2017 at 09:33, Rob Allen <rob@akrabat.com> wrote:
>> >>>
>> >>>> Hi all,
>> >>>>
>> >>>> On the PHP PR, @rr [commented] [1]:
>> >>>>
>> >>>>> The built in packages are convenient - less zip files for the
>> initial
>> >>>> ramp up. But it creates a maintenance issue: when do you pick up
>> updates to
>> >>>> the packages (minor/patch level only?) and not break existing
>> actions using
>> >>>> the "kind". That is: is the kind itself semantically versioned?
>> >>>>
>> >>>> This applies to all kinds and so probably should be discussed project
>> >>>> level and ideally we should document how this is handled.
>> >>>>
>> >>>> There are two things here:
>> >>>>
>> >>>> 1. The language runtimes release patch updates for minor versions.
>> e.g.
>> >>>> PHP `7.1.7` will become `7.1.8` next month with a small number of
>> bug fixes
>> >>>> including crashers and possibly security fixes.
>> >>>>
>> >>>> 2. Each kind bindles a number of packages via the language's standard
>> >>>> package management system: Swift Package Manager for Swift, NPM
for
>> NodeJs,
>> >>>> etc. The projects that produce these packages update them with new
>> versions
>> >>>> minor and patch versions.
>> >>>>
>> >>>> The tension is obviously between keeping updated for fixes vs the
>> risk of
>> >>>> breaks due to a project's inability to keep BC between patch
>> versions. e.g.
>> >>>> the NodeJS kind comes with the `async v2.1.4` package. However
>> `v2.1.5` of
>> >>>> that package fixes a stack overflow issue. Should our actions have
>> that
>> >>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`,
>> but the
>> >>>> latest is `v3.6.0` which is needed for non-experimental API Gateway
>> support…
>> >>>>
>> >>>> Some questions:
>> >>>>
>> >>>> 1. Should we update the language runtime for a kind for a patch
level
>> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
>> latest
>> >>>> `6.9.5`?
>> >>>> 2. Should we ever update the language runtime for a kind for a minor
>> level
>> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the
>> latest
>> >>>> `6.11.1`?
>> >>>> 3. Should we ever update the packages in a kind to the latest patch
>> level
>> >>>> or minor level?
>> >>>> 4. What's our policy when a security issue is published for a
>> language or
>> >>>> a package that we ship in a non-deprecated kind?
>> >>>>
>> >>>> Whatever the answers are, I think we should document them clearly
>> >>>> somewhere.
>> >>>>
>> >>>>
>> >>>> Also, I've started this conversation as a mailing list topic as
it's
>> a
>> >>>> "policy" thing. Given my previous comments on mailing lists, should
>> I also
>> >>>> create a GitHub issue prefixed with "Discussion" to provide more
>> visibility
>> >>>> in order to garner wider community input?
>> >>>>
>> >>>>
>> >>>> Regards,
>> >>>>
>> >>>> Rob...
>> >>>>
>> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
>> >>>> issuecomment-314716101 <https://github.com/apache/
>> >>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Regards,
>> >>> James Thomas
>> >
>> >--
>> >Development thoughts at http://akrabat.com
>> >Daily Jotter for macOS at http://dailyjotter.com
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message