openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Santana <csantan...@gmail.com>
Subject Re: Implementation plan for Go actions
Date Mon, 26 Feb 2018 17:06:11 GMT
Hi Michele

>There is only one "annoyance" here: I noticed the dockerskeleton always
expects a zip file including an "exec"

dockerskeleton suppports adding an exec straight, it will take the file do
a chmod +x and executed.

So not sure what you are referring here as this is possible, maybe I miss
understood


>It is noticeably slow. Go is a bit faster but the problem with Go is
downloading dependencies (that can be a lot...).

This is the major problem for Swift also, SPM (Swift Package Manager) will
take the longest time git cloning entires repos to pull down dependencies

>1. create an apache/incubator-openwhisk-runtime-go repo, so I can fork it
and add my code in order to follow your build infrastructure and code
guidelines. Then I will submit a PR to it.

Yes this was the intention, to create the apache repo, then you can fork it
and submit a PR

>2. ....To make it super easy you may want a simple function somewhere to
import. ....  just to add some code to the existing
https://github.com/apache/incubator-openwhisk-client-go.

This is super goes in line with what we do for the javascript library npm,
we use the same package in action code, and also outside client side to the
whisk api.

>3. Yes no issues in changing the wsk cli

super +1

>  so please let some real expert on swift give a look at what I did and
fix it.

I'm not an expert :-), but I can help here either myself, or I can reach
out to swift experts that I know first hand in IBM from Kitura team





On Mon, Feb 26, 2018 at 10:55 AM Michele Sciabarra <michele@sciabarra.com>
wrote:

>
> On Mon, Feb 26, 2018, at 4:39 AM, Carlos Santana wrote:
> > Great progress Michele +1
> >
> > I agree with the order of your steps.
> >
> > Step #1
> > Let's nock step 1 first.
> > I will summarize it as having first class support for Go at the same
> level
> > of JavaScript, Python, Java, Swift, PHP
> > Meaning from end user support from CLI, docs, runtime image.
> > wsk action create goaction goaction.go
> > wsk action create goaction goexec --kind go:1.10
> > wsk action create goaction goaction.zip --kind go:1.10
> > extend the runtime image with anything you want to add to the image
> > From example:
> > FROM openwhisk/action-go-v1.10
> > apt-get install -y imagemagick
> > docker build . -t csantanapr/action-go-v1.10
> > wsk action create goaction goaction.go --docker
> csantanapr/action-go-v1.10
> > I guess will have a FDK (Functions Developer Kit) :-), making it super
> easy
> > to import it and just write a simple go main method/function the FDK
> > handles the json input, output, etc..
> >
> >
> > I can help by requesting a new runtime repo
> > "apache/incubator-openwhisk-runtime-go"
> > I can setup a boilerplate repo content with some structure for gradle,
> > build, travis, scala tests
> > Then you can submit PR to the repo with go specifics.
> > I guess since your go guru you can handle the few changes to the wsk cli
> > Do you have a repo with some code already I can take a look?
> I have all the code done so far in
> https://github.com/sciabarracom/openwhisk-runtime-go.
> However since I followed my own way, please let do this way:
>
> 1. create an apache/incubator-openwhisk-runtime-go repo, so I can fork it
> and add my code in order to follow your build infrastructure and code
> guidelines. Then I will submit a PR to it.
>
> 2. Regarding the  Go FDK, so far you can easily implement a piped executor
> using standard Go libraries. To make it super easy you may want a simple
> function somewhere to import. I think instead of creating a new sdk the
> best way is just to add some code to the existing
> https://github.com/apache/incubator-openwhisk-client-go.
> I can submit a PR for it.
>
> 3. Yes no issues in changing the wsk cli (I already gave a look at it).
> Will submit PR.
>
>
> >
> > Step #2
> > For swift, I can help since I'm working on swift4, it looks like we can
> > have a new proxy instead of using the python one.
> > We do some pre-compilation/magic today with buildandrecord.py [1] to
> > compile as much as possible up front and creating a shell script that has
> > the exact lines for compiling and linking.
> > Do you have some code on swift consuming from pipe instead of ReadLine I
> > can take a look?
>
> Yes this is the code I am using:
>
> https://github.com/sciabarracom/openwhisk-runtime-go/blob/pipe-loop/hello/swift/Sources/Hello/main.swift
>
> Written this way, it works under my new runtime. Note the ugly hack to
> call the flush in a system dependent way. Unfortunately I have not found a
> portable way to do that, and without flushing the stdout the runtime hangs.
>
> But all my knowledge of Swifts comes from "it looks like a bit like Scala"
> (that I know) and "Google says...", so please let some real expert on swift
> give a look at what I did and fix it.
>
> I think it is possible to have a backward compatible swift runtime maybe
> just running it under the new Go runtime, but before I think I do better to
> have stable code for the Go-only runtime.
>
> >
> > Step #3
> > After doing go and swift, we can see what are the common parts we can
> > extract to be common in the docker runtime repo.
> > And work on backwards compatible single image, and handle the new pipe
> with
> > some annotation, or maybe a new image, we can discuss more deeply once we
> > get there.
> Yup. Once solved Swift it should be easier...
>
> >
> >
> > [1]
> >
> https://github.com/apache/incubator-openwhisk-runtime-swift/blob/master/core/swift41Action/buildandrecord.py
> >
> >
> >
> >
> >
> >
> > On Sat, Feb 24, 2018 at 1:58 PM Michele Sciabarra <
> openwhisk@sciabarra.com>
> > wrote:
> >
> > > Hello, whiskers
> > >
> > > I am at the risk of expanding too much the scope of what I am trying to
> > > do, so I want to try to focus and work step by step.
> > >
> > > So  I am writing down my plan for review.
> > >
> > > # Step one: implement a good support for GoLang, with piped standard
> input
> > > and output, and log in stderr.
> > >
> > > Because there is not a past support for Go, except the generic docker
> > > support, I think I have some freedom here.
> > >
> > > The support will then work this way (please review carefully):
> > >
> > > ## /init
> > >
> > > The runtime expects posted in /init the usual
> > >
> > > {
> > >  "binary": ...
> > >  "code": ...
> > >  "main": ..
> > > }
> > >
> > >  where "code" is either a zip file (with binary: true) or a source code
> > > (with binary: false or absent).
> > >
> > > If it source code, it expects to be a go source.
> > > It places in /action/main/exec.go and compiles it with go build -o
> > > /action/exec /action/main/exec.go, getting a /action/exec to execute.
> > >
> > > If is is a zip file, it will unzip it in /action. If there is a
> > > /action/exec file in the zip it will run it.
> > >
> > > If there is not,  it will look if there is a "main" parameter, getting
> the
> > > <main-value>.
> > >
> > > It will compile a /action/main/<main-value>.go. If the <main-value>
is
> > > missing, it will default to /action/main/exec.go.
> > >
> > > Note the zip can include other source files that will be stored in
> > > /action/<folders>. So I will have to use GOPATH=/action to find them.
> > >
> > > ## /run
> > >
> > > The /run behaviour is:
> > >
> > > At every request, if no action process is running,it will start one and
> > > will pass the first line as an argument and also in standard output.
> > >
> > > If the executable does not terminate, it will keep feeding more input
> in
> > > standard input, expecting output in standard output (line by line) and
> logs
> > > in stderr.
> > >
> > > The server will have to collect stderr logs and write them in docker
> > > stdout, and terminate the output with the markers when an action
> completes:
> > >
> > > XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
> > > XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
> > >
> > > It the executable DOES terminate, it will execute again at the next
> /run
> > > request
> > >
> > > ## Step 2: need to implement the same  logic for swift actions (we
> have to
> > > keep the existing compilation infrastructure)
> > >
> > > Issues to solve here are the compilation and the "warm" startup
> problem. I
> > > have some ideas but... I do better to discuss them once the Go support
> is
> > > finished.
> > >
> > > ## Step 3: will discuss with the list if it is worth to extend the
> support
> > > to the generic dockerskeleton, that is based on the python proxy.
> > >
> > > There are compatibility issues, because the current docker support logs
> > > everything in stdout and the last line is the answer, while for using
> the
> > > pipe you need to do separate log.
> > >
> > > How does it sound?
> > > Suggested or required changes?
> > > Am I missing something?
> > > Approved?
> > >
> > >
> > > --
> > >   Michele Sciabarra
> > >   openwhisk@sciabarra.com
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message