openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Mehrotra <chetan.mehro...@gmail.com>
Subject Re: Re: Backpressure for slow activation storage in Invoker
Date Mon, 24 Jun 2019 09:49:43 GMT
Okie so we can then aim for adding an optional support for storing
activations via a separate service.

Currently we also send the activation result on respective controller
topic. With this change we would also be sending same activation
record on another topic. So we have another choice to make

Option B1 - Activations via controller topic
--------------------------------------------------------

Here we avoid a new topic and instead have a service which listen to
all controller topics for activation records. However that would be
tricky to implement and also tricky to scale out. As scaling out such
a service by running multiple copies would not be easy in terms of
sharding/partitioning

Here the benefit is that we reduce the duplicate writes on Kafka.

Option B2 - Introduce a new topic altogether
-----------------------------------------------------------

We introduce a new topic to which all invokers write the activation
records (like the case for user-events). Then implementing a new
service to read from a single (possibly partitioned topic) would be
easier.

My suggestion is to go for B2 for now.

Any feedback on that?

Chetan Mehrotra

On Fri, Jun 21, 2019 at 11:46 PM Rodric Rabbah <rodric@gmail.com> wrote:
>
> > Can we handle these in same way as user events? Maybe exactly like user
> events, as in use a single service to process both topics.
>
> good call - the user events already contains much of the activation record
> (if not all modulo the logs)?
>
> -r

Mime
View raw message