openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rodric Rabbah <rod...@gmail.com>
Subject Re: factoring out "runtimes" into deployment configuration
Date Fri, 10 Mar 2017 12:14:00 GMT
OK - got it (fwiw that's mostly a CLI tweat to enable it - "2" from earlier
discussions in a longer arc).

-r

On Fri, Mar 10, 2017 at 6:55 AM, Alex Glikson <GLIKSON@il.ibm.com> wrote:

> Yes, zip or code - just like with 'native' actions.
> Think of a case when I am developing a bunch of serverless applications in
> a certain application domain, where certain libraries are a common prereq
> (e.g., opencv). Even if all the dependencies were pure nodejs modules
> (which is not the case for opencv), I would prefer working with my own
> version of the nodejs image and just pass the application code rather than
> pushing large zip files with the same dependencies to all the actions
> (those zip files also have quite restrictive size limit at the moment).
>
> Regards,
> Alex
>
>
>
> From:   Rodric Rabbah <rodric@gmail.com>
> To:     dev@openwhisk.apache.org
> Date:   10/03/2017 01:14 PM
> Subject:        Re: factoring out "runtimes" into deployment configuration
>
>
>
> Ah are you referring to the zip file with "--docker" where you can't name
> the image in the cli today?
>
> -r
>
> > On Mar 10, 2017, at 6:00 AM, Alex Glikson <GLIKSON@il.ibm.com> wrote:
> >
> > I was referring to custom images, which are based on standard ones but
> > with a certain twist (e.g. additional libraries). They would be
> 'blackbox'
> > in terms of image management, but "native" in terms of action code
> > lifecycle. Today you need to use REST API to create/update such actions.
> > Agree that some optimization around #2 would be important to improve
> > performance for such actions.
> > Fine to have popular/approved images in a manifest for now, but would be
>
> > good to have the design flexible enough to enable future replacement
> with
> > a dynamic mechanism (maybe it already is).
> >
> > Regards,
> > Alex
> >
> >
> ============================================================
> ===============
> > Alex Glikson
> > Senior Researcher, Cloud Platforms, IBM Research - Haifa; IBM Lead for
> > FIWARE
> > Email: glikson@il.ibm.com | Skype: alex.glikson | Phone: +972-4-8281085
> |
> > Mobile: +972-54-6466667
> >
> >
> >
> >
> > From:   Rodric Rabbah <rodric@gmail.com>
> > To:     dev@openwhisk.apache.org
> > Date:   10/03/2017 12:32 PM
> > Subject:        Re: factoring out "runtimes" into deployment
> configuration
> >
> >
> >
> > For 1. The cli now is doing the inference for you, sure we can replace
> > that inference with an image name so instead of --kind python you'd
> write
> > --image openwhisk/python. But...
> >
> > The issue more generally however is if you don't solve 2, making it
> easier
> > for users to use images is going to get them penalized for using
> arbitrary
> > but openwhisk-compatible images. So without addressing how to pull
> images
> > to local docker registries at create time for example, this is a non
> > starter in my opinion.
> >
> > The difference with the openwhisk/* images is that they are already
> > pulled, hence that addresses 2 for a limited set of images. These are
> the
> > popular images, per se. If we're missing one, we add it to the manifest.
> >
> > There is still, I posit, a need to know which are the "popular" or
> > approved images to use. That's what the manifest now centralizes into
> the
> > deployment configuration.
> >
> > -r
> >
> >> On Mar 10, 2017, at 2:01 AM, Alex Glikson <GLIKSON@il.ibm.com> wrote:
> >>
> >> I can think of couple of ways to make what you called "kind-aware
> >> blackbox" actions more developer-friendly:
> >> 1. Support them in the CLI (i.e., supporting both "image" and "code"
> >> arguments). This would make the developer experience straightforard.
> >> 2. Generalize the caching/pooling mechanism to handle *popular* images,
>
> >> rather than a hard-coded set of "native" kind images. This would
> > eliminate
> >> the performance overhead.
> >>
> >> Regards,
> >> Alex
> >>
> >> P.S. Once we do #1 and #2 above, my claim is that we don't need
> >> pre-defined set of "kind" images any more, and just need a flexible
> >> hierarchy of images that anyone can reuse and extend
> >>
> >> Markus Thömmes <markusthoemmes@me.com> wrote on 10/03/2017 12:48:17 AM:
> >>
> >>> From: Markus Thömmes <markusthoemmes@me.com>
> >>> To: dev@openwhisk.apache.org
> >>> Date: 10/03/2017 12:48 AM
> >>> Subject: Re: factoring out "runtimes" into deployment configuration
> >>>
> >>> What you're referring to is basically a "kind-aware" blackbox
> >>> action, where you get the easyness of injecting your code and the
> >>> flexibility of using whatever you want in the image as long as the
> >>> interface fits.
> >>>
> >>> I generally like the idea and brought it up as well once (i custom
> >>> built a blackbox to do something similar). Note though that the same
> >>> performance implications as with blackbox images apply.
> >>>
> >>> Von meinem iPhone gesendet
> >>>
> >>>> Am 09.03.2017 um 23:43 schrieb Dascalita Dragos <ddragosd@gmail.com>:
> >>>>
> >>>> With the "runtimeManifest" property I'm wondering if we can also
> >> expose the
> >>>> name of the container image instead of having to compute it in the
> >> code ?
> >>>> Extending the thought: what if we allow users to specify a custom
> >> container
> >>>> name when creating / updating actions, in case users want to ?
> >>>>
> >>>> I'm thinking at the use-case when a package with a group of actions
> >>>> requires multiple libraries for each action. To simplify my example
> >> I'm
> >>>> going to refer only to JS actions:
> >>>> -   Today: when actions needs extra libs users have 2 options :
> >>>>   a) create a zip
> >>>>   b) browserify/"compile" JS into a single JS file
> >>>> -   What I'm thinking: allow users to create their own "runtime"
> image
> >>>> where they can bundle those dependent libs into. All the actions in
> >> the
> >>>> package would be smaller, they would use less space and would
> probably
> >
> >> flow
> >>>> easier through the system when being invoked. A potential problem in
> >> this
> >>>> case would be whether it's secure to let users push their own images
> >> in the
> >>>> OW nodes, but at the same time allowing them to upload a ZIP is
> >>>> conceptually also a "package"/"container" that can contain anything;
> I
> >>>> guess the security aspect would still remain even with user-defined
> >>>> containers.
> >>>>
> >>>> Dragos
> >>>>
> >>>>
> >>>>
> >>>>> On Tue, Mar 7, 2017 at 4:20 AM Michael Marth <mmarth@adobe.com>
> >> wrote:
> >>>>>
> >>>>> Hi Rodric,
> >>>>>
> >>>>> I think that?s great.
> >>>>>
> >>>>> Only comment (more to Matt than you):
> >>>>> If the runtimes are dynamic by deployment (and of course over time)
> >> this
> >>>>> probably should be reflected in the packaging format spec [1].
> >>>>> (see also previous comment [2]:)
> >>>>>
> >>>>> - line 252: not sure if the list of runtime names should be in the
> >>>>> normative
> >>>>> section of a spec, because (as you also write) this list will
> >> fluctuate
> >>>>> between
> >>>>> deployments and over time. OTOH there should be established well
> >> known
> >>>>> names for
> >>>>> well-known runtimes. Maybe point to a community-curated, separate
> >> markdown
> >>>>> file
> >>>>> (where new entries get a PR that can be voted upon)?
> >>>>>
> >>>>> Cheers
> >>>>> Michael
> >>>>>
> >>>>> [1]
> >>>>> https://github.com/openwhisk/openwhisk-wskdeploy/blob/master/
> >>> specification/openwhisk_v0.8.pdf
> >>>>> [2] http://markmail.org/message/pa35wxxl52tvbfxc
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 05/03/17 17:39, "Rodric Rabbah" <rodric@gmail.com<mailto:
> >>>>> rodric@gmail.com>> wrote:
> >>>>>
> >>>>> I've been thinking about how we can add more action runtimes more
> >> easily -
> >>>>> without changing the backend (API, controller, invoker, CLI). One
> way
> >
> >> I'm
> >>>>> leaning toward and started prototyping to get a better understanding
>
> >> of the
> >>>>> feasibility is to strip all runtime types from the backend (and
CLI)
>
> >> and
> >>>>> encode the information about what action runtimes the system
> supports
> >>>>> through configuration parameters.
> >>>>>
> >>>>> This will make it possible to decouple deploying the core components
>
> >> from
> >>>>> managing the action runtime images. An example of encoding the
> >> runtime
> >>>>> information in a deployment configuration is
> >>>>> https://github.com/openwhisk/openwhisk/pull/1948.
> >>>>>
> >>>>> In support of this general direction (even if we settle on a
> >> different
> >>>>> approach), I have increasingly flattened the "Exec" type hierarchy
> in
> >
> >> the
> >>>>> backend[1-3]
> >>>>>
> >>>>> [1] https://github.com/openwhisk/openwhisk/pull/1938
> >>>>> [2] https://github.com/openwhisk/openwhisk/pull/1942
> >>>>> [3] https://github.com/openwhisk/openwhisk/pull/1911
> >>>>>
> >>>>> A sketch of how to use the runtime manifest can be seen in this
WIP
> >> commit:
> >>>>>
> >>>>> https://github.com/rabbah/openwhisk/commit/
> >>> a8890abe51a3e7e6a34f96a46798de660583e36f
> >>>>>
> >>>>> I can see how using this we can add new versions of Node.js,
> >> python:3, and
> >>>>> other handy images by curating a list of these in a separate
> process.
> >>>>>
> >>>>> Feedback, as always, solicited and appreciated.
> >>>>>
> >>>>> -r
> >>>>>
> >>>>>
> >>>
> >>
> >>
> >
> >
> >
> >
> >
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message