openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michele Sciabarra <mich...@sciabarra.com>
Subject Re: Implementation plan for Go actions
Date Mon, 26 Feb 2018 06:38:52 GMT

On Mon, Feb 26, 2018, at 4:39 AM, Carlos Santana wrote:
> Great progress Michele +1
> 
> I agree with the order of your steps.
> 
> Step #1
> Let's nock step 1 first.
> I will summarize it as having first class support for Go at the same level
> of JavaScript, Python, Java, Swift, PHP
> Meaning from end user support from CLI, docs, runtime image.
> wsk action create goaction goaction.go
> wsk action create goaction goexec --kind go:1.10
> wsk action create goaction goaction.zip --kind go:1.10
> extend the runtime image with anything you want to add to the image
> From example:
> FROM openwhisk/action-go-v1.10
> apt-get install -y imagemagick
> docker build . -t csantanapr/action-go-v1.10
> wsk action create goaction goaction.go --docker csantanapr/action-go-v1.10
> I guess will have a FDK (Functions Developer Kit) :-), making it super easy
> to import it and just write a simple go main method/function the FDK
> handles the json input, output, etc..
> 
> 
> I can help by requesting a new runtime repo
> "apache/incubator-openwhisk-runtime-go"
> I can setup a boilerplate repo content with some structure for gradle,
> build, travis, scala tests
> Then you can submit PR to the repo with go specifics.
> I guess since your go guru you can handle the few changes to the wsk cli
> Do you have a repo with some code already I can take a look?
I have all the code done so far in https://github.com/sciabarracom/openwhisk-runtime-go.
However since I followed my own way, please let do this way:

1. create an apache/incubator-openwhisk-runtime-go repo, so I can fork it and add my code
in order to follow your build infrastructure and code guidelines. Then I will submit a PR
to it. 

2. Regarding the  Go FDK, so far you can easily implement a piped executor using standard
Go libraries. To make it super easy you may want a simple function somewhere to import. I
think instead of creating a new sdk the best way is just to add some code to the existing
 https://github.com/apache/incubator-openwhisk-client-go.
I can submit a PR for it.

3. Yes no issues in changing the wsk cli (I already gave a look at it). Will submit PR.


> 
> Step #2
> For swift, I can help since I'm working on swift4, it looks like we can
> have a new proxy instead of using the python one.
> We do some pre-compilation/magic today with buildandrecord.py [1] to
> compile as much as possible up front and creating a shell script that has
> the exact lines for compiling and linking.
> Do you have some code on swift consuming from pipe instead of ReadLine I
> can take a look?

Yes this is the code I am using:
https://github.com/sciabarracom/openwhisk-runtime-go/blob/pipe-loop/hello/swift/Sources/Hello/main.swift

Written this way, it works under my new runtime. Note the ugly hack to call the flush in a
system dependent way. Unfortunately I have not found a portable way to do that, and without
flushing the stdout the runtime hangs.

But all my knowledge of Swifts comes from "it looks like a bit like Scala" (that I know) and
"Google says...", so please let some real expert on swift give a look at what I did and fix
it.
 
I think it is possible to have a backward compatible swift runtime maybe just running it under
the new Go runtime, but before I think I do better to have stable code for the Go-only runtime.

> 
> Step #3
> After doing go and swift, we can see what are the common parts we can
> extract to be common in the docker runtime repo.
> And work on backwards compatible single image, and handle the new pipe with
> some annotation, or maybe a new image, we can discuss more deeply once we
> get there.
Yup. Once solved Swift it should be easier...

> 
> 
> [1]
> https://github.com/apache/incubator-openwhisk-runtime-swift/blob/master/core/swift41Action/buildandrecord.py
> 
> 
> 
> 
> 
> 
> On Sat, Feb 24, 2018 at 1:58 PM Michele Sciabarra <openwhisk@sciabarra.com>
> wrote:
> 
> > Hello, whiskers
> >
> > I am at the risk of expanding too much the scope of what I am trying to
> > do, so I want to try to focus and work step by step.
> >
> > So  I am writing down my plan for review.
> >
> > # Step one: implement a good support for GoLang, with piped standard input
> > and output, and log in stderr.
> >
> > Because there is not a past support for Go, except the generic docker
> > support, I think I have some freedom here.
> >
> > The support will then work this way (please review carefully):
> >
> > ## /init
> >
> > The runtime expects posted in /init the usual
> >
> > {
> >  "binary": ...
> >  "code": ...
> >  "main": ..
> > }
> >
> >  where "code" is either a zip file (with binary: true) or a source code
> > (with binary: false or absent).
> >
> > If it source code, it expects to be a go source.
> > It places in /action/main/exec.go and compiles it with go build -o
> > /action/exec /action/main/exec.go, getting a /action/exec to execute.
> >
> > If is is a zip file, it will unzip it in /action. If there is a
> > /action/exec file in the zip it will run it.
> >
> > If there is not,  it will look if there is a "main" parameter, getting the
> > <main-value>.
> >
> > It will compile a /action/main/<main-value>.go. If the <main-value>
is
> > missing, it will default to /action/main/exec.go.
> >
> > Note the zip can include other source files that will be stored in
> > /action/<folders>. So I will have to use GOPATH=/action to find them.
> >
> > ## /run
> >
> > The /run behaviour is:
> >
> > At every request, if no action process is running,it will start one and
> > will pass the first line as an argument and also in standard output.
> >
> > If the executable does not terminate, it will keep feeding more input in
> > standard input, expecting output in standard output (line by line) and logs
> > in stderr.
> >
> > The server will have to collect stderr logs and write them in docker
> > stdout, and terminate the output with the markers when an action completes:
> >
> > XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
> > XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
> >
> > It the executable DOES terminate, it will execute again at the next /run
> > request
> >
> > ## Step 2: need to implement the same  logic for swift actions (we have to
> > keep the existing compilation infrastructure)
> >
> > Issues to solve here are the compilation and the "warm" startup problem. I
> > have some ideas but... I do better to discuss them once the Go support is
> > finished.
> >
> > ## Step 3: will discuss with the list if it is worth to extend the support
> > to the generic dockerskeleton, that is based on the python proxy.
> >
> > There are compatibility issues, because the current docker support logs
> > everything in stdout and the last line is the answer, while for using the
> > pipe you need to do separate log.
> >
> > How does it sound?
> > Suggested or required changes?
> > Am I missing something?
> > Approved?
> >
> >
> > --
> >   Michele Sciabarra
> >   openwhisk@sciabarra.com
> >

Mime
View raw message