openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erez Hadad" <ER...@il.ibm.com>
Subject Re: Shared Context in OpenWhisk
Date Thu, 06 Dec 2018 16:08:51 GMT
Hi Olivier,

Thank you. The presentation is available at the link posted by Priti.
Let me try to explain what I mean by "orthogonal". A shared context is 
propagated along a flow as it happens. It does not prescribe the flow 
itself, it only evolves with it, over invocations and rules. On the other 
hand, a conductor action implements a certain flow of execution, as 
defined in its code. Therefore, adding a feature that relies on shared 
context (tracing, shared console, stack trace, ... - see presentation) 
using conductor actions requires instrumenting all conductor actions with 
this context - not likely feasible. We cannot foresee all the possible 
applications of shared context beyond the 7-8 options I listed in the 
presentation, so we cannot instrument all the conductor actions to support 
all of them. On the other hand, I'm not sure generic shared context can be 
implemented by conductor actions due to limitations already mentioned - 
not covering rules, not sharing information inside derived actions, only 
conductor can inject additional context. So, each mechanism (conductor 
actions and shared context) has independent value, and should stand on its 
own, independently of the other. 
I agree that some features of shared context can be implemented to some 
degree in conductor actions. However, the implementation would probably be 
unique to conductor actions (as an external orchestrator) and the results 
are not quite the same. For example, a stack trace can be managed by the 
conductor action, but I'm not sure what good it is, because the stack is 
always at depth of 2 (conductor action + current derived action). If a 
derived action invokes another action internally, sync/async/rule, I don't 
see how the conductor stack trace can capture this, because it cannot 
inject anything to the derived action to capture the internal call. 
Agreed ;)
Implementation cost - not sure it's that great, but I need to study the 
issue more. I put one slide in the backup section of the presentation 
listing the main components involved, and this could be further discussed. 
For example, I think that environment variables are provided as a package 
to the runtimes, so adding one more env. var. should make no code 
difference in the runtimes. Alternatively, if using external storage 
(e.g., Redis) for all the context, the key can simply be the activation 
id, which is already there, so no code change at either invoker or 
runtimes. A separate library or service can implement the access to the 
context.
The only suggestion I made for the structure of the shared context is 
using key/value pairs, allow different features to be piggybacked 
independently in the same shared context - for example, enable both 
tracing, unified console and QoS scheduling, each using its own keys. 
Beyond that, implementers of each feature can choose how design their 
data. For example, stack trace can be a flat identifier (later combined 
with time stamps to infer order), or contain the activation id of the 
caller to actually form the stack using pointers, or many other options. 

Regards,
-- Erez





From:   "Olivier Tardieu" <tardieu@us.ibm.com>
To:     dev@openwhisk.apache.org
Date:   05/12/2018 19:39
Subject:        Re: Shared Context in OpenWhisk



Erez,

I did not see an attachment or a link in your email.

---

Some thoughts about the relationship between contexts and conductor 
actions.

1. Contexts are very useful. Love it!

2. Conductor actions depend on managing contexts. So contexts and 
conductors are not orthogonal concerns.

3. Conductor actions make it possible to hide unnecessary context from an 
invoked action but preserve the context so it is still available 
downstream. So one can leverage conductor action for _some_ context 
management including managing a context stack in an invocation tree.

4. Conductor actions piggy back on the parameter object to manage contexts 

and rely on conventions about reserved parameter names. Having a separate 
context table (key/value pair) would have clear advantages. No 
disagreement here. :)

5. For conductor actions, we considered implementing a context separate 
from the parameter object but chickened out basically because it touches a 

lot of OpenWhisk components. This potentially affects all the runtimes, 
the messaging, the activation records, the payload and record sizes, the 
controller memory footprint, the quotas...

6. Conductor actions give the flexibility to customize the context 
management because we can write arbitrary action code to manage context 
propagation. Should we manage the context as a stack? Should it be flat? 
Should an asynchronous invocation preserve or drop the current context? It 

all depends on use cases. If the context becomes a pure runtime facility 
however, implicitly managed and propagated, we loose some of that 
flexibility. So we need to make sure that we identify and support all the 
important patterns beforehand (including obviously conductor action 
contexts).

Cheers,

Olivier




From:   "Erez Hadad" <EREZH@il.ibm.com>
To:     dev@openwhisk.apache.org
Date:   12/05/2018 11:58 AM
Subject:        Shared Context in OpenWhisk



Hi folks,

Following today's call, here is the presentation. Note the additional 
implementation details in the "Backup" section.

Please comment!



Regards,
-- Erez

Erez Hadad, PhD
Cloud System Technologies
IBM Research - Haifa
email: erezh@il.ibm.com
phone: +972-4-829-6509









Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message