openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tyson Norris <>
Subject Re: Improving support for UI driven use cases
Date Sat, 01 Jul 2017 17:53:52 GMT

Point taken on the broad topic :)  On that, I want to point out that the concurrency issue
is currently entangled with the controller discussion, because sequential processing is enforced
(and therefore concurrent processing prevented) at multiple layers, so for better or worse,
it is a bit tricky to talk about them completely in isolation. That said, I agree it will
be good to get some focus on the individual items - I will make an attempt to itemize this
list and the components implicated for each. 


> On Jul 1, 2017, at 10:41 AM, Markus Thömmes <> wrote:
> Thanks for the veeeery detailed writeup! Great job!
> One thing I'd note: As Rodric pointed out we should break the issues apart and address
them one by one.
> For instance the proposed loadbalancer changes (Controllers knows of Containers downstream)
is desirable for each workload and not necessary bound to the core of the proposal, which
I'd say is the concurrency discussion. I agree that 100% warm container usage is crucial there,
but each load will benefit from high container reuse.
> Just not to get the discussion too broad and unfocused.
> On the topic itself: I think its a great idea to further push the performance and, as
pointed out, make operating OpenWhisk more efficient. Most issue I see have already been pointed
out so I won't repeat them. But I'm quite sure we can figure those out.
> Have great weekend!
> Von meinem iPhone gesendet
>> Am 01.07.2017 um 19:26 schrieb Tyson Norris <>:
>> RE: separate policies - I agree that it would make sense for separating container
pools in some way by “event driven” and “ui driven” - I don’t think anything precludes
that from happening, but its different than the notion of “long running” and “short
running”. e.g. if events flow into the system, the container would be equally long running
as if users are continually using the system. I’m not suggesting changing the cold/warm
start behavior, rather the response and concurrency behavior, to be more in line with UI drive
use cases. In fact the “first user” experience would be exactly the same, its the “99
users that arrive before the first user is complete” experience that would be different.
(It may also be appealing to have a notion of “prehot” containers, but I think existing
“prewarm” is good enough for many cases). If it's useful to cap the pool usage for either
of these cases, nothing prevents that from happening, but I would start with a (simpler) case
where there is.a single pool that supports both usages - currently there is no pool that reliably
supports the ui case, since “bursty” traffic is immediately excessively latent.
>> RE: concurrent requests resource usage: I would argue that you would determine resource
allocation in (nearly) the same way you should with single-tasked containers. i.e. the only
pragmatic way to estimate resource usage is to measure it. In single task case, you might
use curl to simulate a single user. In concurrent tasks case, you might use wrk or gatling
(or something else) to simulate multiple users. Regardless of the tool, analyzing code etc
will not get close enough to accurate measurements compared to empirical testing.
>> RE: motivation compared to having a number of hot containers - efficiency of resource
usage for the *OpenWhisk operator*. No one will be able to afford to run open whisk if they
have to run 100 containers *per action* to support a burst of 100 users using any particular
action. Consider a burst of 1000 or 10000 users, and a 1000 actions. If a single container
can handle the burst of 100 users, it will solve a lot of low-medium use cases efficiently,
and in the case of 10000 users, running 100 containers will be more efficient than the 10000
containers you would have to run as single task.
>> WDYT?
>> Thanks for the feedback!
>> Tyson
>> On Jul 1, 2017, at 9:36 AM, Alex Glikson <<>>
>> Having different policies for different container pools certainly makes
>> sense. Moreover, enhancing the design/implementation so that there is more
>> concurrency and less bottlenecks also sounds like an excellent idea.
>> However, I am unsure specifically regarding the idea of handling multiple
>> requests concurrently by the same container. For example, I wonder how one
>> would determine the desired resource allocation for such container?
>> Wouldn't this re-introduce issues related to sizing, scaling and
>> fragmentation of resources - nicely avoided with single-tasked containers?
>> Also, I wonder what would be the main motivation to implement such a
>> policy compared to just having a number of hot containers, ready to
>> process incoming requests?
>> Regards,
>> Alex
>> From:   Rodric Rabbah <<>>
>> To:<>
>> Cc:     Dragos Dascalita Haut <<>>
>> Date:   01/07/2017 06:56 PM
>> Subject:        Re: Improving support for UI driven use cases
>> Summarizing the wiki notes:
>> 1. separate control and data plane so that data plane is routed directly
>> to
>> the container
>> 2. desire multiple concurrent function activations in the same container
>> On 1, I think this is inline with an outstanding desire and corresponding
>> issues to take the data flow out of the system and off the control message
>> critical path. As you pointed out, this has a lot of benefits - including
>> one you didn't mention: streaming (web socket style) in/out of the
>> container. Related issues although not complete in [1][2].
>> On 2, I think you are starting to see some of the issues as you think
>> through the limits on the action and its container and what that means for
>> the user flow and experience. Both in terms of "time limit" and "memory
>> limit". I think the logging issue can be solved to disentangle
>> activations.
>> But I also wonder if these are going to be longer running "actions" and
>> hence, the model is different: short running vs long running container for
>> which there are different life cycles and hence different scheduling
>> decisions from different container pools.
>> [1]
>> [2]

View raw message