openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sugandha Agrawal <sugandha.agrawa...@gmail.com>
Subject Re: A plan to (re) implement OpenWhisk on top of Knative
Date Wed, 03 Jul 2019 14:38:24 GMT
Cool!!!


Regards
Sugandha Agrawal




On Wed, Jul 3, 2019 at 4:37 PM Michele Sciabarra <michele@sciabarra.com>
wrote:

> Thanks! Indeed it was my next step since I am now trying to build actions
> with the actionloop based runtime with tekton. I thought to use Priti work
> that has to be updated to use Tekton so your article is very welcome and
> useful.
>
> --
>   Michele Sciabarra
>   michele@sciabarra.com
>
> ----- Original message -----
> From: Martin Henke <martin.henke@web.de>
> To: dev@openwhisk.apache.org
> Subject: Re: A plan to (re) implement OpenWhisk on top of Knative
> Date: Wednesday, July 03, 2019 4:29 PM
>
> Michele,
>
> FYI: Sugandha and myself published a Medium blog article which describes
> to build Nodejs10 images using Tekton and run it on Knative
> (based on the work from Priti).
> It might be on interest in the context of your Actionloop related Knative
> work.
>
> Link:
>
> https://medium.com/@sugandha.agrawal18/build-knative-service-with-tekton-and-apache-openwhisk-node-js-runtime-f660bbc3a11e
>
> Regards,
> Martin
>
>
> On 2019/05/20 15:00:02, "Michele Sciabarra" <m...@sciabarra.com> wrote:
> > Ok great, I see the discussion is starting to bring ideas.>
> >
> > Yes my goal is basically to run existing actions in Knative, create and
> invoke. And possibile retain the ability of an action to invoke another
> action.>
> >
> > I understand the different way they expose services, so I am rethinking
> the idea of using a "work-alike" path. >
> >
> > If it is needed we can add it with an ingress but it may be not
> necessary in the initial implementation.>
> >
> > Also I checked a bit ML and discussions and I see this Tekton thing that
> should be the preferred way.>
> >
> > Not sure if I understand the relation with the current Build API
> documented in the website. Is Tekton "compatible" or it has a different
> API?>
> >
> >
> > -- >
> >   Michele Sciabarra>
> >   michele@sciabarra.com>
> >
> > ----- Original message ----->
> > From: "Markus Thömmes" <ma...@apache.org>>
> > To: dev@openwhisk.apache.org>
> > Subject: Re: A plan to (re) implement OpenWhisk on top of Knative>
> > Date: Monday, May 20, 2019 4:50 PM>
> >
> > Good discussion, thanks!>
> >
> > Can we try to define what the desired end-goal is here? I'm a bit
> unclear>
> > what resembling the OpenWhisk API actually buys us.>
> >
> > To me, the desired end-state would be to run OpenWhisk actions as-is on
> a>
> > Knative cluster (similar to OpenFaaS' and Azure's integration). There's
> no>
> > good way for us to provide the full API without spinning up a control
> plane>
> > and we can only handle so much via the CLI. So to me, the end-goal
> looks>
> > like:>
> >
> > 1. *wsk action create* actually doing all the pieces necessary to run a>
> > piece of code on Knative.>
> > 2. *wsk action invoke* doing some HTTP call under the hood to "invoke"
> that>
> > action. The action should be reachable via a sensible URL. If we really>
> > want to keep the API surface (as I said, I'm dubious here) we can also
> do>
> > that via ingress level abstractions (like VirtualService).>
> >
> > Cheers,>
> > Markus>
> >
> > Am Mo., 20. Mai 2019 um 15:33 Uhr schrieb Martin Henke <
> martin.henke@web.de>
> > >:>
> >
> > >>
> > > > On 20. May 2019, at 14:55, Michele Sciabarra <mi...@sciabarra.com>>
> > > wrote:>
> > > >>
> > > >> Michele,>
> > > >>
> > > >> I like the idea to make the ActionLoop based runtimes to be
> runnable on>
> > > Knative.>
> > > >>>
> > > >> My thoughts on this:>
> > > >> - I second Markus concern to implement the invocation API onto
> Knative>
> > > instead of just using Knative service syntax.>
> > > > Can you elaborate this? I do not understand.>
> > >>
> > > Knative service syntax:    https://<service(in our case =>
> > > action)>.<namespace>.<host>/>
> > > OW invocation https://>
> > > <api-host>/api/v1/namespaces/<namespace>/actions/<action>>
> > >>
> > > (I personally so no worth in inventing a distinct API for OW images,
> but>
> > > as said I would see that as a valid optional feature)>
> > >>
> > > >>
> > > >> - I would have concerns to make it dependent on Gloo which is kind
> of a>
> > > minority choice for Knative load balancing>
> > > > I do not think it will be hard to setup a test also using Istio, I
> do>
> > > not want to be limited to Gloo.>
> > >>
> > > I just wanted to prevent that Gloo gets a “official” prerequisite for
> an>
> > > “official” OW on Knative flow.>
> > > It is of course free to you to use what ever you want to do in your>
> > > prototype.>
> > >>
> > > > - In my opinion the goal should be to have some uniform behaviour
> for>
> > > ActionLoop based runtimes>
> > > >> and other ones like the adapted NodeJS runtimes demonstrated by
> Matt>
> > > and Priti>
> > > > As much as I can tell the current implementation is just the
> building>
> > > and exposing the "/init" and "/run" but I can be wrong.>
> > > > The build can be of course reused, so it continues the effort. For
> the>
> > > frontend, from the documentation I think Matt wants to add a proxy,
> while I>
> > > would like to implemeent the "invocation" straight in the runtime.
> This is>
> > > open to discussion, but of course it is better to reach an agreement.>
> > >>
> > > Also in the work of Priti and Matt the invocation goes directly to
> the>
> > > runtime. The action code is either passed with the call (not yet
> tested by>
> > > me) or set via environment variable in the docker build.>
> > >>
> > > >>
> > > >> - As Knative Build seems be on a dead end I would propose to
> target>
> > > Tekton as the build system (which developed as kind of >successor out
> of>
> > > Knative)>
> > > >>
> > > > If Knative build is dead then it would be a bit unfair that they
> change>
> > > it as the scope of the Knative project!>
> > > > It looks like the goal is  to setup some standards! And I would be
> very>
> > > disappointed to know that.>
> > >>
> > > Tekton evolved out of Knative Build (or more correct out of Knative>
> > > Pipelines) but is very similar to the Knative build.>
> > > Flows can easily be ported from one to the other,>
> > > If we target Tekton build we would target the platform were the
> Knative>
> > > build team is focusing on.>
> > > But again feel free to use whatever platform for your prototype work.>
> > >>
> > > > At this stage the build is the more interesting thing, and it could
> be>
> > > even imported in main openwhisk to speed up deployment.>
> > > > I have already baked it in the ActionLoop runtimes (with
> precompilation).>
> > > > Also if we use Tekton, where is the Knative standard then? What is
> the>
> > > point? We can build our own system instead of "Knativizing" it...>
> > > >>
> > > >> Maybe it would be a good solution to tackle two things
> independently.>
> > > >> 1) Design and implement a common protocol of building, running and>
> > > calling OW runtimes on Knative>
> > > >> 2) Implement the OW invocation API on top of Knative as an
> additional>
> > > option for those who have the need to expose it.>
> > > >>
> > > > On this, for my personal approach at building things, I want
> something>
> > > that works and it is complete and useful. A "MVP”.>
> > >>
> > > Cool. Just go on.>
> > >>
> > > > So I do not plan to split the effort. Version 0.1 must be a minimal>
> > > working subset of OpenWhisk on Knative.>
> > > > Because otherwise there will be incomplete useless inusable pieces>
> > > around (see for example kwsk).>
> > > >>
> > > > It does not mean that things cannot be modular, nor that everyone
> must>
> > > but to me "openwhisk-knative" must be a single repo with all the
> pieces to>
> > > make something where you can download is and deploy in a kubernetes
> cluster>
> > > and be able to deploy simple actions. When this works, we can improve>
> > > incrementally and split it but keeping it working.>
> > > >>
> > > >> I would looking forward to work with you on the first work item.>
> > > > Great  but I see now more details to discuss before we can start.
> Most>
> > > notably I need to understand how I can build on top of Mark and Priti
> work>
> > > and continue their work. ANd I can even probably recover some of the
> code>
> > > of kwsk as they implemented some openwhisk api, that I want now in
> the>
> > > runtime.>
> > > >>
> > >>
> > > I do not want to stop you in any way. My hope is that the action loop>
> > > runtimes and the “other ones” do expose the same behaviour when being>
> > > called. So that the users is not surprised when calling different
> actions>
> > > in different languages.>
> > > And behaving the same way might also mean to adapt the “other
> languages”>
> > > to the same behaviour as the action loop based ones.>
> > > They just should be uniform to be used.>
> > >>
> > > When your prototype is accessible it would be a good point of time to>
> > > discuss this.>
> > >>
> > > As said I very much like your effort.>
> > >>
> > > >>
> > > >> On 20. May 2019, at 08:55, Michele Sciabarra <mi...@sciabarra.com>>
>
> > > wrote:>
> > > >>>
> > > >>>
> > > >>>> I have an idea for implementing a prototype of OpenWhisk on
top
> of>
> > > Knative.>
> > > >>>> My basic ideas are: do not use any proxy, forwarding or adapter:>
> > > extend>
> > > >>>> the runtime to support the REST call and expose them as ingress.
> And>
> > > use a>
> > > >>>> wrapper on top of `kubectl` to generate all the needed
> components.>
> > > >>>
> > > >>> Does this tie into the work that Matt was doing to the runtimes
to
> make>
> > > >>> them runnable on Knative? Is this lined up with that at all?>
> > > >> Actually yes. He suggested I can investigate how to migrate
> ActionLoop>
> > > to port many other languages to Knative.>
> > > >> Also he recommended I add my idea and this is what I am doing.
> Current>
> > > code is, if I am not wrong, a Knative build of the nodejs runtime.>
> > > >>>
> > > >> There has been a number of attempts and proposal to move forward>
> > > OpenWhisk. My idea that to succeed we need something small but that
> just>
> > > works. This is my idea to be able to implement in the shorter time
> frame>
> > > possible an actual subset of OpenWhisk that works and it is truly
> built on>
> > > top of Knative. So I am putting the thing a bit further than Matt
> work.>
> > > >>>
> > > >>>
> > > >>>> My goal is to have a functional work-alike of OpenWhisk built
on
> top>
> > > of>
> > > >>>> Knative, using ActionLoop as a foundation. I will extend
> ActionLoop to>
> > > >>>> support the required REST calls of OpenWhisk.>
> > > >>>>
> > > >>>> I also want to create tool, I will call `wskn`. This tool
will>
> > > initially>
> > > >>>> just a python script, a wrapper on top of `kubectl` as it
will>
> > > generate>
> > > >>>> kubernetes descriptors.>
> > > >>> Why not build this into "wsk" itself? The Azure Functions CLI
as
> an>
> > > example>
> > > >>> supports multiple deployment types like this in one CLI.>
> > > >>>
> > > >> When it will works, yes, of course. But to start, what I really
> need is>
> > > a prototype that can generate kubernetes descripttors to feed to
> kubectl,>
> > > so a  simplee, quick and ditry, separate tool (that I will keep
> together>
> > > the runtime) is all I need for now.>
> > > >>>
> > > >>>>>
> > > >>>> It will support initially just the the action creation and>
> > > invocation, and>
> > > >>>> only synchronous (blocking) behaviour, as all the request
will
> go>
> > > straight>
> > > >>>> to the runtimes. Hopefully also a subset of `package` and>
> > > `activation`.>
> > > >>>> Again triggers, rules, asynchronous for later.>
> > > >>>>>
> > > >>>> The idea is that you will be able to create actions and web
> actions>
> > > that>
> > > >>>> can run existing OpenWhisk actions, at least those with blocking>
> > > behaviour>
> > > >>>> that run with ActionLoop (Go, Java, Python, PHP, Swift, Rust,
> Ruby,>
> > > >>>> Crystal...)>
> > > >>>>>
> > > >>>> Implementation.>
> > > >>>> ==============>
> > > >>>>>
> > > >>>> This is how I plan to implement it.>
> > > >>>>>
> > > >>>> At this stage I want to use just Knative Serving and Knative
> Build,>
> > > using>
> > > >>>> Gloo for the ingress part. I also plan to install a local
Docker>
> > > registry>
> > > >>>> Kubernetes registry, so we do not have to use DockerHub for>
> > > everything. All>
> > > >>>> of this can be done with existing command line tools in a
few
> minutes>
> > > in>
> > > >>>> any running Kubernetes deployment.>
> > > >>>>>
> > > >>>
> > > >>> Why specifying Gloo here? Do you need anything specific from Gloo>
> > > itself?>
> > > >>> If not I'd propose to just keep it on a Knative Serving API
> surface>
> > > level.>
> > > >> I want to build it on top of Knative serving, full stop.
> Currently,>
> > > installing Gloo is pretty easy and is more  lightweight than Istio so
> I>
> > > will use it for my  first implementation.>
> > > >>>
> > > >>>>>
> > > >>>> When I create an action, it will use Knative build that will
> work>
> > > roughly>
> > > >>>> in this way:>
> > > >>>>>
> > > >>>> - create a configmap with the action code>
> > > >>>> - build the actin using ActionLoop precompilation feature
that
> will>
> > > return>
> > > >>>> a zip file including all the needed to run the action>
> > > >>>> - create a new docker image extending the runtime with the
new
> zip,>
> > > using>
> > > >>>> Kaanico>
> > > >>>> - push the image in the local registry>
> > > >>> This feels like a fairly heavyweight process, we should be able
> to>
> > > come up>
> > > >>> with a way to circumvent zipping entirely. Maybe the runtime can
> detect>
> > > >>> that the unzipped content is already there and skip the unzip
> step?>
> > > >>>
> > > >> Actually this is my first idea of how to use Knative build. And is
> not>
> > > complicated: when I create the action, a run a build that includes
> Kanico.>
> > > I generate a Dockerfile on the fly. The docker file uses the action
> runtime>
> > > that already know how to compile a script. And then I save an image.
> I>
> > > already implemented un "autoinit" so just launching the image will
> give a>
> > > runtime ready to run that execute an action already compiled.>
> > > >>>
> > > >>>
> > > >>> I'm fairly hesitant on the usage of a ConfigMap for storing the
> action>
> > > >>> code. It's all stored in the in-cluster etcd instance and it has
> a>
> > > limit of>
> > > >>> 1M. This is at most a stop-gap solution to provide a PoC I think.
> Any>
> > > ideas>
> > > >>> on how to "productize" this?>
> > > >>>
> > > >> ConfigMap can be mounted as files, so it is an easy way  to feed
> an>
> > > action to a build. It is just an easy way to feed the action code to
> the>
> > > Build.>
> > > >>>
> > > >> My initial constraint is that I want just to generate Kubernetes>
> > > descriptors to feed to kubectl.>
> > > >> Of course in the long run I can add some "file upload" storage.>
> > > >>>
> > > >> If I could to this file upload when invoking a build it could ideal
> as>
> > > I do not have to store anything anywhere, just process the code and>
> > > generate a single layer to execute actions to be store in the
> registry.>
> > > >> I will investigate better this area, I understand your concern.>
> > > >>>
> > > >>>>
> > > >>>> At this point you can run the action. ActionLoop will be extended
> to>
> > > >>>> support invocations in the format>
> > > >>>> "/v1/namespaces/namespace/actions/package/action".>
> > > >>> Why bother reimplementing this exact path? To obtain API
> compatibility>
> > > with>
> > > >>> OpenWhisk as it is today?>
> > > >>>
> > > >> I want to implement a subset of the OpenWhisk API on top of
> Knative>
> > > serving.>
> > > >> Knative serving already does the scaling and routing, so what we
> need>
> > > are the "endpoints" to invoke actions.>
> > > >>>
> > > >> Since I do not want to add additional components, not at the first>
> > > stage. Just knative serve and build, the runtime and a controller
> script,>
> > > the runtime is the natural place where to "handle" the API
> invocations,>
> > > since Knative only generates the URL but not anything else.  If I>
> > > understood well, Matt is adding a proxy. I do not want to add a proxy,
> just>
> > > add to the runtime the ability to respond to "API like" calls, at
> least>
> > > those regarding action invocation.>
> > > >>>
> > > >>>> It will do all the decoding required to invoke the action
with
> the>
> > > >>>> expected paramenters (straight invocation thrhoug the actinloop>
> > > protocol,>
> > > >>>> not proxies).>
> > > >>> Does this mean moving all of the Controller's "smartness" about>
> > > incoming>
> > > >>> and outgoing HTTP requests (see the whole WebActions for
> example)?>
> > > >>>
> > > >> At least decoding web actions in the runtime, yes. Knative serving>
> > > already has routing and proxying.>
> > > >> So a true implementation on top of Knative requires IHMO this>
> > > sacrifice. Unless there is a way to keep the controller in a
> "Knative">
> > > compatible way. Open to suggestions here.>
> > > >>>
> > > >>> Each action will then be exposed using an ingress with its
> specific>
> > > >>> invocation path.>
> > > >>>>
> > > >>> If the community agrees with this plan, I would create a repo>
> > > >>> `incubator-openwhisk-knative` to work on it.>
> > > >>>>
> > > >>> Thoughts?>
> > >>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message