From dev-return-1446-archive-asf-public=cust-asf.ponee.io@openwhisk.apache.org Sat Feb 10 16:48:01 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 5209618061A for ; Sat, 10 Feb 2018 16:48:01 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 41A2F160C4D; Sat, 10 Feb 2018 15:48:01 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 86662160C30 for ; Sat, 10 Feb 2018 16:48:00 +0100 (CET) Received: (qmail 37776 invoked by uid 500); 10 Feb 2018 15:47:59 -0000 Mailing-List: contact dev-help@openwhisk.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@openwhisk.apache.org Delivered-To: mailing list dev@openwhisk.apache.org Received: (qmail 37759 invoked by uid 99); 10 Feb 2018 15:47:58 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Feb 2018 15:47:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 85D31C017A for ; Sat, 10 Feb 2018 15:47:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.148 X-Spam-Level: ** X-Spam-Status: No, score=2.148 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id Ocqu0vuStUPl for ; Sat, 10 Feb 2018 15:47:56 +0000 (UTC) Received: from mail-ua0-f176.google.com (mail-ua0-f176.google.com [209.85.217.176]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id C83585F124 for ; Sat, 10 Feb 2018 15:47:55 +0000 (UTC) Received: by mail-ua0-f176.google.com with SMTP id r4so7029195uak.6 for ; Sat, 10 Feb 2018 07:47:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=uQ+zmeI/m80R2dX4zUhNwYODPskmEWSr6jzez8q1TMo=; b=RftO5ivhFesVKPNmKnmOhqri4CNF0tkR7DeD7ObPhGhh+y5RY9O85HnaFukbVOmF8o NuFSaCHWtwKUZIiMsKzd2tuloMkTS9+PhsK6KSz1DDXHVS2aKL+9+jGF3WbJ1ejMSiQR jnaBJ0rgOjg1vzOsIcdkk7u0pGcyeppptXtiTlhM265xARYm9ZlPiOFiWFtqeRC6CaB1 F1WOcPMEw9JPNIHoeBj5/r9ZUsl1GSfWC2+We2SY55k2puJgbJgCpbfyXxmrZIFdVByE moS9OGlqgItfs9ncLrZ3K30QhQ75sG87JUS2Oo9MImPSmxWs4gb08cRCJ4CUV9PiwYTd vZEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=uQ+zmeI/m80R2dX4zUhNwYODPskmEWSr6jzez8q1TMo=; b=gXf4FQeU7Bw673TebIIV4vYjlb+q1B46bKNTl1r1cFn035URGvoYFjHjmfyVRIiuOr 1QUe2ljanoJcKym9Rf1mWy1H4oLdwuSSaXX2llpTqermioQwdokBnYB4pijZESrbMKhG F/oSZLc3ArWBvwNJRVdm9sMLTpgBwAOIuLnA9f87llCkIvXQkH4VQBk0iVypVnCwlvWE qFJv/mGw/U2Wdyu5HxtHxXn7xtTqt8yy2Cbi3AlJd+6eZkJIQ4OVMCBYS5qPBozmYKUF 4UlxyfF59BgCnUYg3ejVSN+sXIfkWUrurDgb9WwarihA3O7diPPny0310djpwv6FSKsH JUJg== X-Gm-Message-State: APf1xPBcRJx3me21CqJ6TRPrFDcpTS8lHemt6Kr9XDMsqdRL5K80RlzA /UYIt4FubzCLIbv0VjtkkZDd1iLNFrknNxABRBM= X-Google-Smtp-Source: AH8x2269QssZkib8CHfNNa0AtPhGwoyP6Lc1IiyutJE68tm5/tokQ/vskAErOsKiM20AtTKpv7+IdxCjeTXRUnZbDSs= X-Received: by 10.176.20.77 with SMTP id c13mr1519066uae.12.1518277669079; Sat, 10 Feb 2018 07:47:49 -0800 (PST) MIME-Version: 1.0 References: <1518274333.958012.1266289064.4138ABF6@webmail.messagingengine.com> In-Reply-To: <1518274333.958012.1266289064.4138ABF6@webmail.messagingengine.com> From: Carlos Santana Date: Sat, 10 Feb 2018 15:47:38 +0000 Message-ID: Subject: Re: A proposal for a more efficient implementation of Go actions To: dev@openwhisk.apache.org Content-Type: multipart/alternative; boundary="001a114541e422d23f0564dd913c" --001a114541e422d23f0564dd913c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Michele I=E2=80=99m not super familiar with go, but I think it goes inline with wha= t we do with. So If I understand correctly your proposal is for the use case that user provide and precompiled the executable. One approach is for the web server that implements /init on port 8080 to run the go executable the go executable in turn be a another web server that implement the /run endpoint One problem I see is the /run running on port 8080 there will be a port conflict. There could be different options To avoid having use another port like 8081 this will required changes in the invoker to know about. I will try to avoid changes on the invoker to start Another option is for the executable that implements the /run to tell the server on /init to shutdown to release the port and then bind the web server to port 8080 then from this point all the invokes will hit the /run on the users executable action implementing the webserver. This can be documented as the standard api for anyone to write this web server executable. This approach can be also be applicable to other compile languages like Swift which I=E2=80=99m currently working which I=E2=80=99m currently imple= menting for Swift 4. I agree providing a user land library/package like you mentioned will be an improvement if user want an easy way to implement the executable implementing the webserver for the /run endpoint. This is a good experiment and can be implemented today in user land using docker action and quickly iterate and get feedback. This approach can be implemented with modifications to the python proxy to handle the /init but not the /run Something that can be use for Swift to improve the mode of passing params via stdin and last line of stdout and have memory with state and connections and preserved improving performance. Similar as the go library can be offer for Swift to implement the web server wrapper. And other languages :-) I will let others shine in their point of view and other alternatives Thank you for your interested in helping with Go support =E2=80=94 Carlos On Sat, Feb 10, 2018 at 9:52 AM Michele Sciabarra wrote: > # How Go Action are currently implemented > > Currently, Go Action are implemented using the generic docker support. > > I investigated the implementation and I found there is a python based > server, listening for /init and /run requests. The /init will collect an > executable and place in the current folder, while the /run will invoke th= e > executable with popen, feeding the input and returning the output as log, > and the last line as the result as a serialized json. > > The problem is spawning a new process for each request sound like a > revival of the CGI. I made some investigation and it is certainly not th= e > most efficient implementation. Basically everyone moved from CGI executi= ng > processes to servers listening fro requests since many many years ago. > > Also I investigated how AWS Lambda supports go, and as I suspected it is = a > completely different story: they implements a server in Go listening and > serving requests. > > The problem here is Python and Node are dynamic scripting languages, whil= e > Go is a compiled language. > I checked also how Node and Python runtimes work: they are all servers, > they receive the code of the function, =E2=80=9Ceval" the code and then e= xecute it > for serving requests. > > Go, generating an executable, cannot afford to do that. We cannot =E2=80= =9Ceval=E2=80=9D > precompiled code. But it is also inefficient to spawn a new process for > each function invocation. > > The solution here is to exec only once, when the runtime receive the > executable of the function, at the /init time. Then you should replace th= e > main executable and serve the /run requests directly in the replaced > executable. Of course this means that the replaced executable should be > able to serve the /init requests too. All of this should go in a library > > # My proposal. > > Support for Go should look like to the following (very similar to Lambda > support, I admit it): > > package main > > import ( > =E2=80=9Cgithub.com/apache/openwhisk-go-support" > ) > > func hello() (string, error) { > return "Hello Whisk!", nil > } > > func main() { > openwhisk.Start(hello) > } > > The magic will be inside the library. > > The Start function will start a web server listening for two requests. > > Posts to /run will invoke some deserialisation code and then invoke the > function. > > Posts to /init will receive an executable, unzip it, look for an "exec" > file and then exec to it (expecting of course the server itself is > implemented using the same library). > > A new Go specific runtime will built easily using the same library. Maybe > with a default action retuning an error. > > If the proposal sounds acceptable I volunteer to implement it. > > -- > Michele Sciabarra > openwhisk@sciabarra.com > --001a114541e422d23f0564dd913c--