openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Thomas <jthomas...@gmail.com>
Subject Re: Updating Package and Language versions for a kind
Date Fri, 28 Jul 2017 16:33:10 GMT
That sounds good. I've been thinking about reviewing the existing packages
and trying to propose a new list based upon real-world usage and relevance.
Looking at the current Node.js list:
https://github.com/apache/incubator-openwhisk/blob/master/core/nodejs6Action/Dockerfile#L33

It has packages like "express" amongst others which I can't see a use for?

On 27 July 2017 at 15:20, Carlos Santana <csantana23@gmail.com> wrote:

> Being thinking more about this, and having a core set of packages included
> by default is very useful and I think critical for open source integration
> and continues integration in the open.
> Also keeping large actions away from the system for folks that just want to
> get started and kick the tires using the embedded packages
>
> I think we could have a mid way alternative:
>
> What about if we create new kinds every time we want to pickup an update
> for new language update but at the minor (y) level (version x.y.z).
> Also include a very small set of core packages, for example request for
> nodejs very stable and popular package
>
> We create a new kind to pick up a new minor update and at that time will
> pickup the latest version of the packages.
> For patch updates, at least for security updates this are done at the patch
> level, will update the current kind to update the patch level of the
> language.
>
> So I think this aligns with what Rob mentioned about trying to keep kinds
> versions at the minor level.
>
> Taking nodejs as an example:
> we have nodejs:6 (this is set to 6.9)
> let's create a new kind for node nodejs:6.11
> This will include latest version of 6.11 and latest versions of packages at
> the time of creating of the new kind nodejs:6.11
> If there are nodejs patch updates to pickup for 6.11 we update without
> creating a new kind will be kept at nodejs:6.11 with the packages pinned to
> the versions that they shipped.
> I think nodejs moves a bit faster than the other languages so we could
> pickup  a new minor nodejs:6.12 or 6.14 when ever the community thinks
> there is a minor update from nodejs worth the update to create a new kind.
>
> There might be a gap, I think it's OK for those corner cases that an user
> wants to use the latest nodejs version but use old version of the packages,
> or use the latest version of the package but an old version of nodejs. I
> think for this cases user would need to pinned and include the exact
> version of the modules to include and that's OK, I think this will be a
> normal thing for people going into production and pinning and controlling
> dependencies.
>
> I think it's hard to come up with silver bullet for all languages, as every
> language updates at a different speed, and dependencies are more slowest
> pace than others.
>
>
>
>
>
>
>
> On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <csantana23@gmail.com>
> wrote:
>
> > One thought I have being debating is that the open source project doesn't
> > include any packages going forward.
> > The project will have an image that is small and common denominator that
> > the operators deploying decide which packages to add on top.
> >
> > For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017),
> >
> > Will keep the version of language nodejs 8 up to date to the latest as
> > official releases come out from nodejs
> > Operators like IBM and Adobe, RH, others, can base their custom images
> for
> > their offerings on the same base image, adding packages that they want
> base
> > on their offering.
> >
> > Same for swift new kind swift:4, and so on.
> >
> > This will be a general policy, as every language has their peculiarity
> and
> > probably we need our runtimes SMEs to talk about the risk on updating or
> > not updating per language.
> >
> > dealing with user level packages is always a pickle for example today we
> > have nodejs:6 and we have a set of users that benefit from updating the
> > packages to the latest latest, and could have users that they depend on a
> > current version of the package and updating the package on them could
> cause
> > problems. If I had my way I would say thought luck for using the embedded
> > package and not packaging or doing a bundle build, but serverless is on
> > baby years of adoption and I don't want to be thought yet :-)
> >
> > So for nodejs8 we say thought luck no embedded packages, but will do
> > something to make it super easy will have the system help you
> build/compile
> > your action and include your dependencies, these system actions that
> allow
> > users to provide only their code/git-url, and package.json/package.swift,
> > and these actions will compile/npm install etc..  and install your action
> > including the declared dependencies.
> >
> >
> >
> >
> >
> > On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <mmarth@adobe.com.invalid>
> > wrote:
> >
> >> Hi Rob,
> >>
> >> Re “I wonder how many deployments roll their own action containers?”
> >>
> >> I spoke too general (implying what most OW deployments would want). What
> >> I can say is that our team wishes to be in control of the libraries
> that go
> >> into the action containers, that’s why we need to roll our own.
> >>
> >> Agree with the approach you have outlined on patch and minor releases.
> >>
> >> The topic of deprecation IMHO is the same discussion as when to upgrade
> >> language runtimes in action containers: at least for us the deprecation
> >> policy is something that we need to be in control of (which is a given
> as
> >> we roll our own containers).
> >> +1 to the deprecation tooling you hint at regarding not creating actions
> >> for deprecated containers (this is needed regardless of any deprecation
> >> policies). This is not covered by the issues you linked to, is it?
> >>
> >> Cheers
> >> Michael
> >>
> >>
> >>
> >> On 14/07/17 19:08, "Rob Allen" <rob@akrabat.com> wrote:
> >>
> >> >Hi Michael, all,
> >> >
> >> >I wonder how many deployments roll their own action containers? I
> >> suspect not many.
> >> >
> >> >I agree with you both that it would be nice to keep packages (modules)
> >> up to date on a regular basis within a kind makes the most sense and a
> >> regular schedule would be nice, especially if there's a documented way
> to
> >> do the update for each kind.
> >> >
> >> >I think we should also keep the language of the kind up to date at the
> >> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 -
> >> unless there are language kinds that break semver at the patch level?
> (In
> >> which case my head may explode!)
> >> >
> >> >For minor releases, it's a little more wooly at the moment at least.
> For
> >> NodeJS, [PR 2078  [1] simply updates the `node6` kind from 6.9.1 to
> 6.11.0
> >> without creating a new new "kind". However, for Swift, [PR] [2120] gave
> us
> >> a new `swift:3.1.1` kind rather than updating the `swift3` kind which
> runs
> >> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though,
> so
> >> maybe it should have been called `swift:3.1`?
> >> >
> >> >My thoughts on this is that we should create new kinds for minor
> >> language releases on the principle of least surprise for the container
> >> users.
> >> >
> >> >I would like to fairly aggressively deprecate old ones to stop new
> >> actions being created for out-of-date language versions, but as
> deprecating
> >> a kind prevents invoking any actions for that kind that's not possible
> >> unless some work is done to allow invoking actions on a deprecated kind,
> >> but not creating or updating them would be necessary for that to be a
> >> possibility.
> >> >
> >> >I think GitHub Issue [319] [3] is relevant in terms of the tooling
> >> around this.
> >> >
> >> >
> >> >Regards,
> >> >
> >> >Rob...
> >> >
> >> >[1]: https://github.com/apache/incubator-openwhisk/pull/2078
> >> >[2]: https://github.com/apache/incubator-openwhisk/pull/2120
> >> >[3]: https://github.com/apache/incubator-openwhisk/issues/319
> >> >
> >> >
> >> >> On 14 Jul 2017, at 16:27, Michael Marth <mmarth@adobe.com.INVALID>
> >> wrote:
> >> >>
> >> >> Rob, all,
> >> >>
> >> >> I largely echo what James wrote below (except moving this discussion
> >> off-list :) )
> >> >>
> >> >> Incidentally, we had a very similar discussion within my team
> recently
> >> on how to handle this topic for our OW deployment. The bottom line is
> that
> >> the trade-off between stability and being up-to-date can only be
> decided in
> >> a given business context of the OW deployment.
> >> >> We (as a project) do not have that context and cannot really make the
> >> decision, IMHO. In that light the only reasonable policy in my view is
> to
> >> strive to keep everything updated as best as we can, like monthly or
> every
> >> second month, etc.
> >> >> If we miss a beat (i.e. don’t immediately update to latest release)
I
> >> don’t think that’d be critical. Like James said, most OW providers will
> >> have their own runtimes anyway and would (should) take the latest if
> they
> >> need it.
> >> >>
> >> >> Cheers
> >> >> Michael
> >> >>
> >> >>
> >> >> On 13/07/17 11:02, "James Thomas" <jthomas.uk@gmail.com> wrote:
> >> >>
> >> >>> Good ideas Rob. I had a similar issue when looking at the Swift
> >> runtime
> >> >>> recently.
> >> >>>
> >> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a
> 23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E
> >> >>>
> >> >>> Would it be possible to have a scheduled upgrade process for
> installed
> >> >>> modules? Once every four, six or eight weeks? If OpenWhisk did
start
> >> to
> >> >>> produce "releases", it could tie in with that.
> >> >>>
> >> >>> I'd guess that most people using the built-in packages are more
> >> kicking the
> >> >>> tires than building production apps. Once you start being a
> >> production app,
> >> >>> you'll want to explicitly bundle and control your app dependencies.
> >> I'd +1
> >> >>> on being more aggressive with upgrading module versions.
> >> >>>
> >> >>> I'd like to have a Github issue to follow for this, I find it easier
> >> than
> >> >>> the mailing list.
> >> >>>
> >> >>> On 13 July 2017 at 09:33, Rob Allen <rob@akrabat.com> wrote:
> >> >>>
> >> >>>> Hi all,
> >> >>>>
> >> >>>> On the PHP PR, @rr [commented] [1]:
> >> >>>>
> >> >>>>> The built in packages are convenient - less zip files for
the
> >> initial
> >> >>>> ramp up. But it creates a maintenance issue: when do you pick
up
> >> updates to
> >> >>>> the packages (minor/patch level only?) and not break existing
> >> actions using
> >> >>>> the "kind". That is: is the kind itself semantically versioned?
> >> >>>>
> >> >>>> This applies to all kinds and so probably should be discussed
> project
> >> >>>> level and ideally we should document how this is handled.
> >> >>>>
> >> >>>> There are two things here:
> >> >>>>
> >> >>>> 1. The language runtimes release patch updates for minor versions.
> >> e.g.
> >> >>>> PHP `7.1.7` will become `7.1.8` next month with a small number
of
> >> bug fixes
> >> >>>> including crashers and possibly security fixes.
> >> >>>>
> >> >>>> 2. Each kind bindles a number of packages via the language's
> standard
> >> >>>> package management system: Swift Package Manager for Swift,
NPM for
> >> NodeJs,
> >> >>>> etc. The projects that produce these packages update them with
new
> >> versions
> >> >>>> minor and patch versions.
> >> >>>>
> >> >>>> The tension is obviously between keeping updated for fixes
vs the
> >> risk of
> >> >>>> breaks due to a project's inability to keep BC between patch
> >> versions. e.g.
> >> >>>> the NodeJS kind comes with the `async v2.1.4` package. However
> >> `v2.1.5` of
> >> >>>> that package fixes a stack overflow issue. Should our actions
have
> >> that
> >> >>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk
v3.3.2`,
> >> but the
> >> >>>> latest is `v3.6.0` which is needed for non-experimental API
Gateway
> >> support…
> >> >>>>
> >> >>>> Some questions:
> >> >>>>
> >> >>>> 1. Should we update the language runtime for a kind for a patch
> level
> >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1`
to the
> >> latest
> >> >>>> `6.9.5`?
> >> >>>> 2. Should we ever update the language runtime for a kind for
a
> minor
> >> level
> >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1`
to the
> >> latest
> >> >>>> `6.11.1`?
> >> >>>> 3. Should we ever update the packages in a kind to the latest
patch
> >> level
> >> >>>> or minor level?
> >> >>>> 4. What's our policy when a security issue is published for
a
> >> language or
> >> >>>> a package that we ship in a non-deprecated kind?
> >> >>>>
> >> >>>> Whatever the answers are, I think we should document them clearly
> >> >>>> somewhere.
> >> >>>>
> >> >>>>
> >> >>>> Also, I've started this conversation as a mailing list topic
as
> it's
> >> a
> >> >>>> "policy" thing. Given my previous comments on mailing lists,
should
> >> I also
> >> >>>> create a GitHub issue prefixed with "Discussion" to provide
more
> >> visibility
> >> >>>> in order to garner wider community input?
> >> >>>>
> >> >>>>
> >> >>>> Regards,
> >> >>>>
> >> >>>> Rob...
> >> >>>>
> >> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415#
> >> >>>> issuecomment-314716101 <https://github.com/apache/
> >> >>>> incubator-openwhisk/pull/2415#issuecomment-314716101>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Regards,
> >> >>> James Thomas
> >> >
> >> >--
> >> >Development thoughts at http://akrabat.com
> >> >Daily Jotter for macOS at http://dailyjotter.com
> >> >
> >>
> >
>



-- 
Regards,
James Thomas

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message