openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Glikson" <GLIK...@il.ibm.com>
Subject Re: factoring out "runtimes" into deployment configuration
Date Fri, 10 Mar 2017 11:55:02 GMT
Yes, zip or code - just like with 'native' actions.
Think of a case when I am developing a bunch of serverless applications in 
a certain application domain, where certain libraries are a common prereq 
(e.g., opencv). Even if all the dependencies were pure nodejs modules 
(which is not the case for opencv), I would prefer working with my own 
version of the nodejs image and just pass the application code rather than 
pushing large zip files with the same dependencies to all the actions 
(those zip files also have quite restrictive size limit at the moment).

Regards,
Alex



From:   Rodric Rabbah <rodric@gmail.com>
To:     dev@openwhisk.apache.org
Date:   10/03/2017 01:14 PM
Subject:        Re: factoring out "runtimes" into deployment configuration



Ah are you referring to the zip file with "--docker" where you can't name 
the image in the cli today?

-r

> On Mar 10, 2017, at 6:00 AM, Alex Glikson <GLIKSON@il.ibm.com> wrote:
> 
> I was referring to custom images, which are based on standard ones but 
> with a certain twist (e.g. additional libraries). They would be 
'blackbox' 
> in terms of image management, but "native" in terms of action code 
> lifecycle. Today you need to use REST API to create/update such actions.
> Agree that some optimization around #2 would be important to improve 
> performance for such actions.
> Fine to have popular/approved images in a manifest for now, but would be 

> good to have the design flexible enough to enable future replacement 
with 
> a dynamic mechanism (maybe it already is).
> 
> Regards,
> Alex
> 
> 
===========================================================================
> Alex Glikson
> Senior Researcher, Cloud Platforms, IBM Research - Haifa; IBM Lead for 
> FIWARE
> Email: glikson@il.ibm.com | Skype: alex.glikson | Phone: +972-4-8281085 
| 
> Mobile: +972-54-6466667
> 
> 
> 
> 
> From:   Rodric Rabbah <rodric@gmail.com>
> To:     dev@openwhisk.apache.org
> Date:   10/03/2017 12:32 PM
> Subject:        Re: factoring out "runtimes" into deployment 
configuration
> 
> 
> 
> For 1. The cli now is doing the inference for you, sure we can replace 
> that inference with an image name so instead of --kind python you'd 
write 
> --image openwhisk/python. But...
> 
> The issue more generally however is if you don't solve 2, making it 
easier 
> for users to use images is going to get them penalized for using 
arbitrary 
> but openwhisk-compatible images. So without addressing how to pull 
images 
> to local docker registries at create time for example, this is a non 
> starter in my opinion. 
> 
> The difference with the openwhisk/* images is that they are already 
> pulled, hence that addresses 2 for a limited set of images. These are 
the 
> popular images, per se. If we're missing one, we add it to the manifest.
> 
> There is still, I posit, a need to know which are the "popular" or 
> approved images to use. That's what the manifest now centralizes into 
the 
> deployment configuration.
> 
> -r
> 
>> On Mar 10, 2017, at 2:01 AM, Alex Glikson <GLIKSON@il.ibm.com> wrote:
>> 
>> I can think of couple of ways to make what you called "kind-aware 
>> blackbox" actions more developer-friendly:
>> 1. Support them in the CLI (i.e., supporting both "image" and "code" 
>> arguments). This would make the developer experience straightforard.
>> 2. Generalize the caching/pooling mechanism to handle *popular* images, 

>> rather than a hard-coded set of "native" kind images. This would 
> eliminate 
>> the performance overhead.
>> 
>> Regards,
>> Alex
>> 
>> P.S. Once we do #1 and #2 above, my claim is that we don't need 
>> pre-defined set of "kind" images any more, and just need a flexible 
>> hierarchy of images that anyone can reuse and extend
>> 
>> Markus Thömmes <markusthoemmes@me.com> wrote on 10/03/2017 12:48:17 AM:
>> 
>>> From: Markus Thömmes <markusthoemmes@me.com>
>>> To: dev@openwhisk.apache.org
>>> Date: 10/03/2017 12:48 AM
>>> Subject: Re: factoring out "runtimes" into deployment configuration
>>> 
>>> What you're referring to is basically a "kind-aware" blackbox 
>>> action, where you get the easyness of injecting your code and the 
>>> flexibility of using whatever you want in the image as long as the 
>>> interface fits.
>>> 
>>> I generally like the idea and brought it up as well once (i custom 
>>> built a blackbox to do something similar). Note though that the same
>>> performance implications as with blackbox images apply.
>>> 
>>> Von meinem iPhone gesendet
>>> 
>>>> Am 09.03.2017 um 23:43 schrieb Dascalita Dragos <ddragosd@gmail.com>:
>>>> 
>>>> With the "runtimeManifest" property I'm wondering if we can also 
>> expose the
>>>> name of the container image instead of having to compute it in the 
>> code ?
>>>> Extending the thought: what if we allow users to specify a custom 
>> container
>>>> name when creating / updating actions, in case users want to ?
>>>> 
>>>> I'm thinking at the use-case when a package with a group of actions
>>>> requires multiple libraries for each action. To simplify my example 
>> I'm
>>>> going to refer only to JS actions:
>>>> -   Today: when actions needs extra libs users have 2 options :
>>>>   a) create a zip
>>>>   b) browserify/"compile" JS into a single JS file
>>>> -   What I'm thinking: allow users to create their own "runtime" 
image
>>>> where they can bundle those dependent libs into. All the actions in 
>> the
>>>> package would be smaller, they would use less space and would 
probably 
> 
>> flow
>>>> easier through the system when being invoked. A potential problem in 
>> this
>>>> case would be whether it's secure to let users push their own images 
>> in the
>>>> OW nodes, but at the same time allowing them to upload a ZIP is
>>>> conceptually also a "package"/"container" that can contain anything; 
I
>>>> guess the security aspect would still remain even with user-defined
>>>> containers.
>>>> 
>>>> Dragos
>>>> 
>>>> 
>>>> 
>>>>> On Tue, Mar 7, 2017 at 4:20 AM Michael Marth <mmarth@adobe.com>

>> wrote:
>>>>> 
>>>>> Hi Rodric,
>>>>> 
>>>>> I think that?s great.
>>>>> 
>>>>> Only comment (more to Matt than you):
>>>>> If the runtimes are dynamic by deployment (and of course over time) 
>> this
>>>>> probably should be reflected in the packaging format spec [1].
>>>>> (see also previous comment [2]:)
>>>>> 
>>>>> - line 252: not sure if the list of runtime names should be in the
>>>>> normative
>>>>> section of a spec, because (as you also write) this list will 
>> fluctuate
>>>>> between
>>>>> deployments and over time. OTOH there should be established well 
>> known
>>>>> names for
>>>>> well-known runtimes. Maybe point to a community-curated, separate 
>> markdown
>>>>> file
>>>>> (where new entries get a PR that can be voted upon)?
>>>>> 
>>>>> Cheers
>>>>> Michael
>>>>> 
>>>>> [1]
>>>>> https://github.com/openwhisk/openwhisk-wskdeploy/blob/master/
>>> specification/openwhisk_v0.8.pdf
>>>>> [2] http://markmail.org/message/pa35wxxl52tvbfxc
>>>>> 
>>>>> 
>>>>> 
>>>>> On 05/03/17 17:39, "Rodric Rabbah" <rodric@gmail.com<mailto:
>>>>> rodric@gmail.com>> wrote:
>>>>> 
>>>>> I've been thinking about how we can add more action runtimes more 
>> easily -
>>>>> without changing the backend (API, controller, invoker, CLI). One 
way 
> 
>> I'm
>>>>> leaning toward and started prototyping to get a better understanding


>> of the
>>>>> feasibility is to strip all runtime types from the backend (and CLI)


>> and
>>>>> encode the information about what action runtimes the system 
supports
>>>>> through configuration parameters.
>>>>> 
>>>>> This will make it possible to decouple deploying the core components


>> from
>>>>> managing the action runtime images. An example of encoding the 
>> runtime
>>>>> information in a deployment configuration is
>>>>> https://github.com/openwhisk/openwhisk/pull/1948.
>>>>> 
>>>>> In support of this general direction (even if we settle on a 
>> different
>>>>> approach), I have increasingly flattened the "Exec" type hierarchy 
in 
> 
>> the
>>>>> backend[1-3]
>>>>> 
>>>>> [1] https://github.com/openwhisk/openwhisk/pull/1938
>>>>> [2] https://github.com/openwhisk/openwhisk/pull/1942
>>>>> [3] https://github.com/openwhisk/openwhisk/pull/1911
>>>>> 
>>>>> A sketch of how to use the runtime manifest can be seen in this WIP 
>> commit:
>>>>> 
>>>>> https://github.com/rabbah/openwhisk/commit/
>>> a8890abe51a3e7e6a34f96a46798de660583e36f
>>>>> 
>>>>> I can see how using this we can add new versions of Node.js, 
>> python:3, and
>>>>> other handy images by curating a list of these in a separate 
process.
>>>>> 
>>>>> Feedback, as always, solicited and appreciated.
>>>>> 
>>>>> -r
>>>>> 
>>>>> 
>>> 
>> 
>> 
> 
> 
> 
> 
> 






Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message