openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michele Sciabarra <>
Subject [LONG] Discussing my implementation of Go actions
Date Fri, 09 Mar 2018 11:13:25 GMT
I just did a  PR of my version of the Golang action implementation. It does some "breaking"
changes  and there is some discussion on the slack channel.  

So I report the current situation n here, looking for advices and change recommendations.
 Since I am a bit confused, if I remember well, one Apache rule is  the mailing list is the
ultimate source for the truth...

It currently works this way (I call it the "pipe-loop" protocol)

A golang action (or a generic binary) is expected to follow this "protocol":

* starts with  {"openwhisk": 1}
* reads on line in stardard input, expecting a json ON A SINGLE LINE
* process the line, emits logs in stderr (can be multiple lines)
* outputs a line in stdout in json format ON A SINGLE LINE
* repeat forever

It is important to note this design is easy to implement and works even for bash scripts,
but it is easy to use also perl, ruby, haskell in an EFFICIENT way.  Indeed this bash script
(with jq) is part of my tests:

echo '{"openwhisk":1}'
while read line
   name="$(echo $line | jq -r .name)"
   logger -s "name=$name" 
   hello="Hello, $name"
   logger -s "sent response"
   echo '{"hello":"'$hello'"}'

Things discussed:

1) ​remove the header {"openwhisk":1}

Actually initially it was not there. But I decided to add this requirements because the action
need to speak a protocol ANYWAY. 

Most important, I explain why I require it starts with "{"openwhisk: 1}".

The main reason is: I start the child process at init time, and I wanted to detect when it
does not behave properly.

The simplest problem happens when the action crashes immediately. For example, a common reason
for this problem is uploading a binary using some dynamic libraries not available in the runtime.
For  example a swift action. By defaults it load a lot of different libraries, it crashes
immediately but I cannot detect it until I try to read its stdin.

I can remove this requirement if someone can show me the go code to check that cmd.Start("true")
or cmd.Start("pwd") exited 😃

If it is not doable, and I skip  the handshake, even if the command crashed, I will not detect
the problem until a /run is executed and the action times out...

Carlos say it is fine. It is ok for me but I still think an early problem detection would
be better. Also James recommended me to provide as much as error detection to the user as
early as possible. Kinda of conflicting directives here...


2) more checks at init time

I added some sanity checks.  Probably too many. I tried to detect the error at deployment
time, not at invocation time.

This is different from what currently for example dockerskeleton does.

If I upload for example something wrong, like a non-zip, a non-elf executable, my init returns
{"error": "description"}, while currently the dockerskeleton returns always OK.

Recommendations here?

3) output to another channel the result

Currently I require logs goes to stderr, and stdout is for interacting with the parent process.

Rodric suggested to output to a separate channel (channel 3?)  and use stdout and stderr for

While doable, I need to provision another pipe, and the implementation should probably do
some syscalls to retrieve file descriptor 3. It would complicate implementation, while currently
it is straightforward for any language that does not have a library available. For swift,
even to flush stdout I needed to write "linux specific" code... I do not dare to think what
I need to do to write in fd3...

My opinion is that using stdout for I/O and stderr for logs is a better choice than opening
another file descriptor. 

Thoughts here?

  Michele Sciabarra

View raw message