From dev-return-3019-archive-asf-public=cust-asf.ponee.io@openwhisk.apache.org Tue Jan 22 12:23:34 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 62800180634 for ; Tue, 22 Jan 2019 12:23:33 +0100 (CET) Received: (qmail 62049 invoked by uid 500); 22 Jan 2019 11:23:32 -0000 Mailing-List: contact dev-help@openwhisk.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@openwhisk.apache.org Delivered-To: mailing list dev@openwhisk.apache.org Received: (qmail 62036 invoked by uid 99); 22 Jan 2019 11:23:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jan 2019 11:23:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 46226C600C for ; Tue, 22 Jan 2019 11:23:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.702 X-Spam-Level: X-Spam-Status: No, score=-0.702 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id Lm4pu7sTR17t for ; Tue, 22 Jan 2019 11:23:29 +0000 (UTC) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 92A2E60F5F for ; Tue, 22 Jan 2019 11:23:28 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id EB8AE210A8 for ; Tue, 22 Jan 2019 06:23:27 -0500 (EST) Received: from web1 ([10.202.2.211]) by compute2.internal (MEProxy); Tue, 22 Jan 2019 06:23:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=yYOZRUcrmRq79/psd7NbPpIhfS7gfWypMKiVTY6/k cg=; b=pA1Ve1UhA4cMHp3RV71s0jSKBpMwUM1c/VgZCygapXR609BEL3YQYr1GA XqyKBXcCa+o7S6/egu5V9r+rpkCT4Xg2ev82bYPz8pElkjEZGxBgBAfiy/Y8tYjs gewILiW7PpAma7Aw4a0H5rChQciHWVNWXrWUoAK+bJx9ti2XrbxYzEsRHfNuu+xQ EB321EuY/EC++HiuJ/7c2H/JmRL+LaWMQzxLmE16i4o/RqmQxlFStSsNVq+I24DD JjyMEecgo0yZdhQPdzWf2jvtBFi3gRliPYrjePDq9HJqoYkvZsKzLE2sPDD6Tw3s A8CXM4j9yxsLWE/2qeUrlq6OHl+dQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtledrheekgddviecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecufedt tdenucenucfjughrpefkhffvggfgtgfofffujghfsehtjeertdertdejnecuhfhrohhmpe foihgthhgvlhgvucfutghirggsrghrrhgruceomhhitghhvghlvgesshgtihgrsggrrhhr rgdrtghomheqnecuffhomhgrihhnpehstghirggsrghrrhgrrdhnvghtpdhgihhthhhusg drtghomhenucfrrghrrghmpehmrghilhhfrhhomhepmhhitghhvghlvgesshgtihgrsggr rhhrrgdrtghomhenucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 5874294211; Tue, 22 Jan 2019 06:23:27 -0500 (EST) Message-Id: <1548156207.2190315.1640721120.0B50A04C@webmail.messagingengine.com> From: Michele Sciabarra To: dev@openwhisk.apache.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-36e4bfd3 Date: Tue, 22 Jan 2019 12:23:27 +0100 Subject: Re: [Long] I implemented websocket support for the Golang and related runtimes: In-Reply-To: <24BB5654-309F-48FA-9868-F9FA3ABB5A46@gmail.com> References: <1547748462.3466746.1637314640.642C3C09@webmail.messagingengine.com> <1548095681.1281286.1640147528.0167AB1A@webmail.messagingengine.com> <24BB5654-309F-48FA-9868-F9FA3ABB5A46@gmail.com> I guess then this feature is of interest and worth to be merged, as it provides a basis on the runtimes to implement websockets as a general feature then. Am I correct ? :) -- Michele Sciabarra michele@sciabarra.com ----- Original message ----- From: Carlos Santana To: dev@openwhisk.apache.org Subject: Re: [Long] I implemented websocket support for the Golang and related runtimes: Date: Tue, 22 Jan 2019 05:49:39 -0500 Thanks Michele for the clarification My knative remark was only about to scale to zero, meaning when there is times that no one is using my app no pod/container is running at all. The main.go code can still be loaded via ConfigMap in Knative or without in Kubernetes. It still uses the same image - Carlos Santana @csantanapr > On Jan 21, 2019, at 1:34 PM, Michele Sciabarra wrote: > > Actually YES this is exactly what I did. A websocket that accept a websocket and writes in kafka. > > There is not yet a demo, but I am working on it. I am building a small webchat doing it. > > So far the demo is only this one https://hellows.sciabarra.net/ that uses the "hello.go" as an action turned in a websocket. > > If you use a websocket client you can see it is actually the action aswering in a websocket. > >> ws wss://hellows.sciabarra.net/hello >> {"name":"Mike"} > < {"golang-main-single":"Hello, Mike!"} > >> {} > < {"golang-main-single":"Hello, world!"} > > You just need the action loop container modified and you can deploy an action in Kubernetes > > Yes this can be hooked in knative, but I did it simpler with an "autoinit". > Code of the action is provided in a file through a configmap and then the runtime initialize the action instead of waiting a /init and serves it in a websocket. > > The deployment of the "https://hellows.sciabarra.net" is entirely here: > > https://github.com/sciabarracom/incubator-openwhisk-runtime-go/blob/websocket-support/examples/websocket-hello/hellows.yml > > The client is just a static html page. > > I can talk of this work wednesday if we can consider merging this feature. > > -- > Michele Sciabarra > michele@sciabarra.com > > ----- Original message ----- > From: Carlos Santana > To: dev@openwhisk.apache.org > Subject: Re: [Long] I implemented websocket support for the Golang and related runtimes: > Date: Mon, 21 Jan 2019 12:37:05 -0500 > > Hi Michele this looks very cool in deed > > Where you able to create a container in k8s with a main.go that took > websocket input and output to a persistent kafka connection > > I'm interested in this as I have a use case that I want IBM Cloud Function > to ingest into kafka, and I was going to build similar thing as you did > using http 1or 2 server, that then ingest into kafka as a service (Event > Streams). > I was going to deploy this container into kubernetes, but I wanted the > container to scale down to zero using Knative. > > You have such main.go for your runtime with websocket at one side and kafka > producer at the other? > > -- Carlos > > On Thu, Jan 17, 2019 at 1:07 PM Michele Sciabarra > wrote: > >> Hello whiskers! >> >> Sorry it is a bit long, so I split it into parts with headlines. >> >> TL;DR >> >> I implemented support for Websocket so you can deploy an Action as a >> WebSocket server if you have a Kubernetes cluster (or just a Docker >> server). See at the end of this post for an example of a Kubernetes >> deployment descriptor. >> >> Here is a very simple demo using it: >> >> https://hellows.sciabarra.net/ >> >> It uses a websocket server implemented using the golang runtime and the >> code of an UNCHANGED OpenWhisk action. All the magic happens at deployment >> using the descriptor provided. >> >> I believe it is a good foundation for implementing websocket support in >> OpenWhisk. The next step would be to provide support at the API level. >> >> After reading the rest, the question is: does the community approve this >> feature? If yes, I can submit a PR for including it in the next release of >> the actionloop. >> >> 1. Motivation: why I did this >> >> A few days ago I asked what was the problem in having an action that >> creates a persistent connection to Kafka. I was answered that a Serverless >> environment can flood Kafka with requests because more actions are spawn >> when the load increase. >> >> The solution is to create a separate server to be deployed somewhere, for >> example, Kubernetes, maybe using WebSockets to communicate with Kafka. In >> short, I had the need to transform an action in a WebSocket server. >> >> Hence I had the idea of adding WebSocket support to Action as WebSocket >> server, adding support for WebSocket to ActionLoop, so I could create a >> WebSocket server in the same way as you write an action. >> >> 2. What I did >> >> I implemented WebSocket support in the action runtime. If you deploy the >> action now it answers not only to `/run` but also to `/ws` (it is >> configurable) as a WebSocket in continuous mode. >> >> You enable the WebSocket setting the environment variable OW_WEBSOCKET. >> Also, for the sake of easy deployment, there is also now an autoinit >> feature. If you set the environment variable to OW_AUTOINIT, it will >> initialize the runtime from the file you specified in the variable. >> >> Ok, fine you can say, but how can I use it? >> >> With a Kubernetes descriptor! You can launch the runtime in Kubernetes, >> provide the main action in it (you can also download it from a git repo or >> store in a volume), and now your action is a web socket server answering to >> your requests. >> >> Look to the following descriptor for an example: >> >> It is a bit long, this is what it does >> >> - it creates a configmap containing the action code >> - it launches the image mounting the action code >> - the image initialize the action and then listen to the WebSocket >> - the image also exposes the web socket using an ingress >> >> >> apiVersion: v1 >> kind: Namespace >> metadata: >> name: hellows >> --- >> apiVersion: v1 >> kind: ConfigMap >> metadata: >> name: hellows >> namespace: hellows >> data: >> main.go: | >> package main >> import "fmt" >> func Main(obj map[string]interface{}) map[string]interface{} { >> name, ok := obj["name"].(string) >> if !ok { >> name = "world" >> } >> fmt.Printf("name=%s\n", name) >> msg := make(map[string]interface{}) >> msg["golang-main-single"] = "Hello, " + name + "!" >> return msg >> } >> --- >> apiVersion: v1 >> kind: Pod >> metadata: >> name: hellows >> namespace: hellows >> labels: >> app: hellows >> spec: >> volumes: >> - name: mnt >> configMap: >> name: hellows >> containers: >> - name: hellows >> image: actionloop/golang-v1.11:ws3 >> ports: >> - containerPort: 8080 >> protocol: TCP >> volumeMounts: >> - name: mnt >> mountPath: "/mnt" >> env: >> - name: OW_WEBSOCKET >> value: /hello >> - name: OW_AUTOINIT >> value: /mnt/main.go >> --- >> apiVersion: v1 >> kind: Service >> metadata: >> name: hellows >> namespace: hellows >> spec: >> ports: >> - port: 8080 >> protocol: TCP >> targetPort: 8080 >> selector: >> app: hellows >> --- >> apiVersion: extensions/v1beta1 >> kind: Ingress >> metadata: >> name: hellows >> namespace: hellows >> spec: >> rules: >> - host: hellows.sciabarra.net >> http: >> paths: >> - path: /hello >> backend: >> serviceName: hellows >> servicePort: 8080 >> >> >> -- >> Michele Sciabarra >> michele@sciabarra.com >> > > > -- > Carlos Santana >