openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erez Hadad" <ER...@il.ibm.com>
Subject Re: Recording metadata related to activation
Date Wed, 21 Aug 2019 10:05:06 GMT
On the same note, why not also expose this "meta" information to the 
action code *at runtime*? 
The current direction this discussion is going seems to be having the 
"meta" information only after the action completes, in an activation 
record (under new key or as annotations).

However, think of the following use-case: the "transaction id" can be 
useful for having multiple actions performing computation as part of a 
single transaction, and updating a DB. In such a case, the action code 
needs to know the transaction id so it can be passed to the DB service, 
marking the resulting update as part of the broader transaction. 
Similar cases can be made for other fields. 

Bottom line: I think this "meta" information needs to be more streamlined 
end-to-end, available to code during invocation and persisted post-factum 
in the activation record.

Regards,
-- Erez




From:   Dominic Kim <style9595@gmail.com>
To:     dev@openwhisk.apache.org
Date:   21/08/2019 02:58
Subject:        [EXTERNAL] Re: Recording metadata related to activation



That would be useful from the operator point of view.
One question is "would that information be exposed to users"?

I think the information which is exposed to users should be
platform-independent.
No matter which underlying platform/implementation is being used, users do
and should not need to know about the internal.
So that even if the operator changes their internals(K8s, native, cluster
federation, ...) there should be no difference in user experience.

One option can be storing them as parts of an activation for operators but
exclude them when returning them in response to the user request.
Though I am not sure whether this can be aligned with what you keep in 
your
mind.


Regarding the two structure options, I am inclined to use the existing
structure "annotations" as it does not introduce any schema change.
However, I also found it cumbersome to manipulate them in many cases.
I feel it would be great to change annotations to a dictionary at some
point.

Since I am not aware of the history, I am curious whether there is any
specific reason that annotations should be the current form.

Best regards
Dominic

2019년 8월 21일 (수) 오전 12:38, Matt Sicker <boards@gmail.com>님이 작성:

> I mean, unless you're using these correlation ids in your business
> logic, I don't see the problem of storing them in the database. My own
> thoughts on using this feature would all be diagnostics-related. I'm
> not running any non-trivial functions, though.
>
> On Tue, 20 Aug 2019 at 05:30, Chetan Mehrotra 
<chetan.mehrotra@gmail.com>
> wrote:
> >
> > Hi Team,
> >
> > Branching the thread [1] to discuss how to record some metadata
> > related to activation. Based on some of the usecases I see a need to
> > record some more metadata related to activation. Some examples are
> >
> > 1. transactionId - Record the transactionId for which the activation 
is
> part of
> > 2. pod name - Records the pod running the action container when using
> > KubernetesContainerFactory
> > 3. invocationId - Some id returned by underlying system when
> > integrating with AWS Lambda or Azure Function
> > 4. clusterId - If running multiple clusters for same system we would
> > like to know which cluster handed the given execution
> >
> > Some of these ids are determined as part of `ContainerResponse` itself
> > and have to be made part of activation json such that later we can
> > correlate the activation with other parts.
> >
> > Now we need to determine how to store such id
> >
> > Option 1 - New "meta" sub document
> > -----------
> >
> > Introduce a new "meta" key in activation json under which we store 
such
> ids
> >
> > "meta" : {
> >             "transactionId" : "xxx",
> >             "podId" : "ow_xxx"
> >         }
> >
> >
> > Option 2 - Store them as annotations
> > -------------
> >
> > Instead of  introducing a new field we store them as annotations. Note
> > we still make change in code to capture such data as part of
> > `ContainerResponse` but just map it to annotations
> >
> > One drawback of this approach is that current approach of annotations
> > make it harder to index such fields easily. Having a flat structure
> > like with "meta" field enables indexing such fields in db's other than
> > Couch
> >
> > Chetan Mehrotra
> > [1]:
> 
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.apache.org_thread.html_f8b73a9ffb0d09a50aecfb54538da2e8365c54dcc3e26a78382ad7bd-40-253Cdev.openwhisk.apache.org-253E&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Oo9B0p_tCCWIIum5GpjjqA&m=45iTZS1Qpqk36WKRpGn5AVuVekJOiQscMAmVG6gb-Ao&s=TkpxifEFrs94Fzs57UTthhI93j1M6QOyBvfqkyirmXg&e=


>
>
>
> --
> Matt Sicker <boards@gmail.com>
>




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message