From dev-return-4033-archive-asf-public=cust-asf.ponee.io@openwhisk.apache.org Wed Jun 26 18:27:03 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id E6EBC18067C for ; Wed, 26 Jun 2019 20:27:02 +0200 (CEST) Received: (qmail 91245 invoked by uid 500); 26 Jun 2019 18:27:02 -0000 Mailing-List: contact dev-help@openwhisk.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@openwhisk.apache.org Delivered-To: mailing list dev@openwhisk.apache.org Received: (qmail 91136 invoked by uid 99); 26 Jun 2019 18:27:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Jun 2019 18:27:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 7589DC1627 for ; Wed, 26 Jun 2019 18:27:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.803 X-Spam-Level: * X-Spam-Status: No, score=1.803 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id sf4ReDYrTydM for ; Wed, 26 Jun 2019 18:26:59 +0000 (UTC) Received: from mail-yw1-f68.google.com (mail-yw1-f68.google.com [209.85.161.68]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 299DF5F56F for ; Wed, 26 Jun 2019 18:26:58 +0000 (UTC) Received: by mail-yw1-f68.google.com with SMTP id b143so1651079ywb.7 for ; Wed, 26 Jun 2019 11:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=vgkTCXNyamQ7ZzJW9UrW55i2KBSpkJmutZXhImu5wKI=; b=RKn+AcVt4QGktlYtCTImMgECUBos/OpNdvRWWcW+AtBCdB3NPv8rKL4TS7Fq4l2PL8 TO6/gjwfYg8vnsYUgSgZfX3IlNT3v0I3zyj67u+GZ7LKaVwZBEU/BKH62mKzi9eG8mt+ LKIdpZmPir2Q1Za4cT2Xf74WddE9mKcagOclmK1va2llkHWmg9Sqa/u+rkNk3X5hC0bL AaZuF9/hEYVoukTen07ayq+15N8ngT9PON3GWMz794XJQ1kOmUa22ElQjjPX80vgT8AW nuaSxfcSKzWiBrGHrwXgtzIlL2WkzIOwJk9eEkBiUn0MzMF1fkfu82nnDN3+pxxlI6Op i9wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=vgkTCXNyamQ7ZzJW9UrW55i2KBSpkJmutZXhImu5wKI=; b=dOVkNmx+ws36WsLQCwuH2oQ5/IvZsS/ul3kVD/xVtX787pjw18qivvpSpbDFhraRqX 13nCwtj07Ateva6gLMGkZ+LnZSjokcxA2QHmSTjFCl8T/lpTTEvl8u4QH3l6ips0lfbH b+SbP1pJvpQfSHg0GUfT2By+fGQTtB5Ptey5JibPAqRUD7+wijrmTsEeF+UIn6y0ISV3 W8eAPw8Q/DLMJGFNk0LM/ekVKao3s6dao6DtfNWY/E7twp95XTRfLOduoVp4OU1qMhRp d1GtDDEgD8vYGRuaVAUvq/qzcqfQZYZXQ8xcoMN2KJoKA9z6XeVpe4hFhnstYXaaikKM JB6Q== X-Gm-Message-State: APjAAAXkjKir70poHz47Yoi8qdoFDxAZqjKo2pM2I4mF8c+qrOhNt+j3 E4PYnI9Yzu/HgpNGkc9y9zp3vQZJhnwTSfTBSShCTBBE X-Google-Smtp-Source: APXvYqyCm9T/CqdE7KaZakhn5zxheawy7KcHbt/TyPQnsVSegH4Godjb1U+mN2S1F/mN4Vzs3N4AQ1hWsX0ifhj8lmI= X-Received: by 2002:a81:1d05:: with SMTP id d5mr3615364ywd.299.1561573616570; Wed, 26 Jun 2019 11:26:56 -0700 (PDT) MIME-Version: 1.0 References: <643CE0B6-E18B-42C1-B3D0-6AB4D1A0FABF@adobe.com> <8EF411CF-92B7-4CD0-8387-890825C22D54@gmail.com> <014167B4-E60A-4C39-A3FC-11B965D01F79@adobe.com> In-Reply-To: <014167B4-E60A-4C39-A3FC-11B965D01F79@adobe.com> From: Rodric Rabbah Date: Wed, 26 Jun 2019 14:26:20 -0400 Message-ID: Subject: Re: exporting activation arguments to the environment To: dev@openwhisk.apache.org Content-Type: multipart/alternative; boundary="000000000000b4d8cc058c3e307e" --000000000000b4d8cc058c3e307e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > Sorry, it still seems controversial to me, not sure how others feel? That's why we discuss on the dev list :D Thanks for the feedback so far. > can you confirm this is decided based on the case of the parameter name? Indeed, we need some rule to then partition the parameter list. Using the convention that the env var starts with a capital letter is one. Other conventions are plausible. > adding a '-e' flag that specifically does "set these environment variables" Sure - but this increases the complexity of implementation significantly for not a lot of gain. To add a -e, we'd need to modify the schema for actions. For example, we could add annotation for each parameter name to be treated as an environment variable using the existing annotations, and use these annotations as the criteria. We could create a new field in the actions object to hold the parameters (a schema change). We could annotate each parameter (also a schema change). Since a developer already controls the names of their parameters today, they have complete control over this partitioning. If we're open to schema changes, then we can explore a cleaner implementation but an incremental approach that at least makes the feature available incrementally would also make sense since making a schema change is a lot more invasive, coupled with a few changes needed at the invoker level plus all the runtimes. -r On Wed, Jun 26, 2019 at 2:07 PM Tyson Norris wrote: > Sorry, it still seems controversial to me, not sure how others feel? > > To be clear, when you added "-a partition-arguments true", the result is > 2 things: > 1. some of the -p args are now treated different than others - can you > confirm this is decided based on the case of the parameter name? > 2. init receives these params (which sounds good to me). > > Regardless of opting in to this behavior, having action-configured > parameters referenced differently based on the name of the param seems > bad. I understand there are some useful conventions defining these as en= v > vars, but my point is that this doesn't seem at all like an explicit > choice. I think an explicit choice would be more like adding a '-e' flag > that specifically does "set these environment variables", instead of > overloading the '-p' flag with a convention based on the name of the > variable. > > Thanks > Tyson > > > =EF=BB=BFOn 6/26/19, 10:43 AM, "Rodric Rabbah" wrote: > > Maybe this got missed, but here's how I conceived of this. I'll use > wsk CLI > commands, I think it makes it obvious. > > wsk action create myAction code.js -p MY_ENV true -p some_param false > -a > partition-arguments true > > The annotation (partition-arguments) makes it explicit for the > developer to > control whether "main" receives the arguments as they do today, which > is > this object > { MY_ENV: true, some_param: false}, or when the annotation is true, { > some_param: false} and process.env.MY_ENV is set to true. > > I don't think there's anything confusing about this in that the > developer > has decided what variables to export to the environment, and is makin= g > an > explicit choice. > > Environment variables on a number of platforms are restricted to thos= e > at > consist of words that start with capital letter (AWS, Netlify as two > prime > examples). > > The alternative, today, requires a function to export any variables > from > "main" to the environment. So it would explicitly export MY_ENV to th= e > environment. The change we're discussing frees the programmer from > having > to do that. > > The change to the runtime proxies would be 1. accept an additional > value on > /init and export all the properties it contains to the environment. > > Before I address the POST invoke issue, I'd like to make sure my > explanation is clearer and if this is still controversial. > > -r > > > On Wed, Jun 26, 2019 at 1:21 PM Tyson Norris > > wrote: > > > Are you saying one is exported to environment, during init, based o= n > > parameter name being UPPER case? Forgetting use of env vars for a > minute, > > this seems confusing to treat parameters different based on names. = I > would > > rather see either a) all action-configured params sent to init only= , > and > > never to run or b) all action-configured params sent to run as > context > > object. > > > > What the runtime does at init (use env vars or not) can be differen= t > per > > runtime, but in the action-configured parameter case I don't see an= y > > problem with setting env vars, except that there seems to be a > convention > > in some cases that allows invoking clients to "override" these > values using > > POST parameters at invocation time. This also seems confusing but > could > > also be enforced differently by various runtimes, although ideally = I > would > > rather see the convention change to: action-configured parameters a= re > > always sent to init, and always visible to run, regardless of what > client > > sends as execution parameters. > > > > Thanks > > Tyson > > > > > > On 6/25/19, 3:32 PM, "Rodric Rabbah" wrote: > > > > Context and Knative I view as orthogonal. > > > > That is, for the context object, it is another way of > encapsulating > > arguments. It doesn=E2=80=99t export variable to the process enviro= nment. > > > > You can provide an action with both environment variables, > arguments > > to main, and a context object. They are orthogonal. > > > > For the context object, the distinction that was necessary from > > previous discussions was related to separating intra container > concurrent > > executions. If the system-provided context is exported to the > environment > > as it today the values clobber each other. For this, the context > object > > would make sense. > > > > I=E2=80=99m simply talking about two parameters wsk ... =E2=80= =9C-p a A=E2=80=9D and =E2=80=9C-p > B b=E2=80=9D > > say where one becomes exported to the environment as B=3Db and the > other is > > passed to the action as ({a:A}). > > > > I=E2=80=99m going to set the knative discussion aside because I= think > it=E2=80=99s a > > distraction. With knative you can bind environment variables to the > > container. As you would with any other container. > > > > I think it=E2=80=99s too simplistic to say knative has a single= endpoint. > > After all there are readiness probes and possible pre/post start > hooks that > > operators may have to deal with. Init can be viewed as the readines= s > probe. > > > > Fundamentally I believe the actor model is much better aligned > with > > the reactive programming model for functions so this will tend > toward a > > completely different discussion in my view. > > > > The reason my proposal sets the environment variables at init > time is > > that=E2=80=99s how env vars work; they exist before you start you = process. > While > > they don=E2=80=99t need to be immutable, it makes sense to test the= m as such. > > > > For webaction parameters that one would export to an > environment, they > > are already immutable and cannot be overridden. So really you would > not use > > them for anything that varies per activation. > > > > The view here is that you can export global (immutable) > variables to > > the action. This makes it easier to take existing code and > containers which > > might use env vars and use them almost off the shelf. > > > > -r > > > > > On Jun 25, 2019, at 6:07 PM, Tyson Norris > > > wrote: > > > > > > I had to read this several times, but have some suggestions. = I > think > > when you say "action's arguments", you mean action-configured > params, e.g. > > `wsk action create --param p1 v1`? > > > > > > My preferences would be: > > > - we should split off "run" args into context and params - > this is > > the convention change for redefining main(args) as main(context, > args) we > > have discussed in the past. > > > - I support either having init receive action-configured para= ms > > > - activation args that are possibly overridden should behave > exactly > > as specified args - is it important that action-configured args are > > actually overridden, if the context and params are separated? > (receive both > > values, and logic must decide when to use which) > > > - let's not use env variables for any arg that is variable pe= r > > activation - it is impossible if you support concurrency, and > unneeded if > > we pass the context to "run". > > > > > > Regarding Matt's suggestion to remove init - I like this idea= , > but I > > have concerns compared to knative which might serve every function > with a > > different container, vs having some containers reused for multiple > > functions. In the case where we init code into an already running > > container, it is useful to have the init process separate from run, > since > > otherwise each runtime will need to track its own init state and > queue > > requests during init etc. If I'm not getting the whole picture with > > knative, please correct me. > > > > > > > > > Thanks > > > Tyson > > > > > > On 6/24/19, 8:43 AM, "Rodric Rabbah" wrote= : > > > > > > In the current activation model, an action's arguments are > always > > provided > > > to the action on "run", not "init". > > > > > > Should we consider partitioning the argument list into two > sets, > > the first > > > is exported as environment variables at "init" time, and t= he > > second become > > > the action's argument at "run" time? A criteria for > partitioning > > is that > > > the environment variable starts with a capital letter, > which is a > > common > > > convention. > > > > > > For example, an action which is invoked with a JSON object > > > > > > { "XYZ": true, > > > "abc" : false } > > > > > > would receive {"abc": false} as its arguments and can read > XYZ > > from the > > > environment (as process.env.XYZ =3D=3D "true" in Node.js). > > > > > > This change would: > > > 1. require a change in the invoker to pass arguments durin= g > > initialization > > > > > > 2. require a change in the runtime proxies to export the > > arguments to the > > > environment at initialization time (additional work may be > > implied by 1b) > > > > > > 3. an annotation on actions to opt into this partitioning > for > > backward > > > compatibility or to opt out. For example '-a > > env-partition-arguments true' > > > partitions the arguments and actions without this > annotation are > > not > > > affected. > > > > > > Some obvious question: > > > Q1a. should the invoker perform the partitioning or > delegate it > > to the > > > runtime? The advantage of the former is that the runtimes > do not > > have to > > > implement the filtering policy and do less work. I think i= t > makes > > sense to > > > do this invoker side for uniformity. > > > > > > Q1b. should the partitioning treat environment variables a= s > > immutable post > > > init and ignore the partition on warm starts? This is an > issue > > when a value > > > is overridden during POST invoke only since for a > webaction, you > > cannot > > > override a value that's already defined (and updating a > bound > > parameter on > > > an action invalidates warm containers). I think env vars > should > > be treated > > > as immutable despite the issue with POST invoke. > > > > > > -r > > > > > > > > > > > > > > > --000000000000b4d8cc058c3e307e--