From dev-return-1655-archive-asf-public=cust-asf.ponee.io@openwhisk.apache.org Fri Mar 9 12:13:38 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9307718064A for ; Fri, 9 Mar 2018 12:13:37 +0100 (CET) Received: (qmail 26012 invoked by uid 500); 9 Mar 2018 11:13:36 -0000 Mailing-List: contact dev-help@openwhisk.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@openwhisk.apache.org Delivered-To: mailing list dev@openwhisk.apache.org Received: (qmail 25997 invoked by uid 99); 9 Mar 2018 11:13:35 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Mar 2018 11:13:35 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 76CADC012E for ; Fri, 9 Mar 2018 11:13:35 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.702 X-Spam-Level: X-Spam-Status: No, score=-0.702 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 7mX-oxU5V8e0 for ; Fri, 9 Mar 2018 11:13:33 +0000 (UTC) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id A31405F503 for ; Fri, 9 Mar 2018 11:13:32 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 2EA7321846 for ; Fri, 9 Mar 2018 06:13:26 -0500 (EST) Received: from web2 ([10.202.2.212]) by compute2.internal (MEProxy); Fri, 09 Mar 2018 06:13:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=hZcwBfj/qTnFKHSbUVOCGOJjvVtjV FXyKpZlQtqKGYo=; b=gtL15N2IHT8rOv5YwYZjpZy0GUZ825KZ32hAgJzTd2nu/ ypP3r7aty1kHf0RBRDTs2u10Rc/VqA3BTJyaEH+LCqhnoERiKLWpm7hU84A74pVw t9QKyasYYizMkdizuTvXCcWplRtjZrGgNClvWj3a7KxDVzF8We4YVaF9oV5WyDQT 2uURU8JVZSMHhG1Kg1/0p/YEG6DB6mn49hTvzF9kgER1bxIHIearwLnGMG0CDKWF ol2qgh5/J+55uvO6BgnYNuCCEesTuK8mYr6+elf1PliOAFPAMUmhT8tM6SyDDG41 xMHdYBJ1sbTOPR7TXAhvxJSnXgRGGUP8pBCRr+CIA== X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id F231C621BF; Fri, 9 Mar 2018 06:13:25 -0500 (EST) Message-Id: <1520594005.1059955.1297179240.0E81AC97@webmail.messagingengine.com> From: Michele Sciabarra To: dev@openwhisk.apache.org MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-54087d22 Date: Fri, 09 Mar 2018 12:13:25 +0100 Subject: [LONG] Discussing my implementation of Go actions I just did a PR of my version of the Golang action implementation. It does= some "breaking" changes and there is some discussion on the slack channel= .=20=20 So I report the current situation n here, looking for advices and change re= commendations. Since I am a bit confused, if I remember well, one Apache r= ule is the mailing list is the ultimate source for the truth... It currently works this way (I call it the "pipe-loop" protocol) A golang action (or a generic binary) is expected to follow this "protocol": * starts with {"openwhisk": 1} * reads on line in stardard input, expecting a json ON A SINGLE LINE * process the line, emits logs in stderr (can be multiple lines) * outputs a line in stdout in json format ON A SINGLE LINE * repeat forever It is important to note this design is easy to implement and works even for= bash scripts, but it is easy to use also perl, ruby, haskell in an EFFICIE= NT way. Indeed this bash script (with jq) is part of my tests: --- #!/bin/bash echo '{"openwhisk":1}' while read line do name=3D"$(echo $line | jq -r .name)" logger -s "name=3D$name"=20 hello=3D"Hello, $name" logger -s "sent response" echo '{"hello":"'$hello'"}' done --- Things discussed: 1) =E2=80=8Bremove the header {"openwhisk":1} Actually initially it was not there. But I decided to add this requirements= because the action need to speak a protocol ANYWAY.=20 Most important, I explain why I require it starts with "{"openwhisk: 1}". The main reason is: I start the child process at init time, and I wanted to= detect when it does not behave properly. The simplest problem happens when the action crashes immediately. For examp= le, a common reason for this problem is uploading a binary using some dynam= ic libraries not available in the runtime. For example a swift action. By = defaults it load a lot of different libraries, it crashes immediately but I= cannot detect it until I try to read its stdin. I can remove this requirement if someone can show me the go code to check t= hat cmd.Start("true") or cmd.Start("pwd") exited =F0=9F=98=83 If it is not doable, and I skip the handshake, even if the command crashed= , I will not detect the problem until a /run is executed and the action tim= es out... Carlos say it is fine. It is ok for me but I still think an early problem d= etection would be better. Also James recommended me to provide as much as e= rror detection to the user as early as possible. Kinda of conflicting direc= tives here... Suggestions? 2) more checks at init time I added some sanity checks. Probably too many. I tried to detect the error= at deployment time, not at invocation time. This is different from what currently for example dockerskeleton does. If I upload for example something wrong, like a non-zip, a non-elf executab= le, my init returns {"error": "description"}, while currently the dockerske= leton returns always OK. Recommendations here? 3) output to another channel the result Currently I require logs goes to stderr, and stdout is for interacting with= the parent process. Rodric suggested to output to a separate channel (channel 3?) and use stdo= ut and stderr for logs.=20 While doable, I need to provision another pipe, and the implementation shou= ld probably do some syscalls to retrieve file descriptor 3. It would compli= cate implementation, while currently it is straightforward for any language= that does not have a library available. For swift, even to flush stdout I = needed to write "linux specific" code... I do not dare to think what I need= to do to write in fd3... My opinion is that using stdout for I/O and stderr for logs is a better cho= ice than opening another file descriptor.=20 Thoughts here? --=20 Michele Sciabarra openwhisk@sciabarra.com