openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dascalita Dragos <ddrag...@gmail.com>
Subject Re: Supporting user-configurable warm action containers?
Date Thu, 31 May 2018 19:45:41 GMT
Acknowledging that dev cycles, performance tests, or other spiky traffic
may not be initially captured, I would put my $$$ on teaching OW how to
learn the traffic pattern and let it pre-warm based on that. There are
existing algorithms that could be used for prediction, if we just have the
features/data captured. This is a beautiful area, yet unchartered territory
so far ?

A related aspect is "how many VMs should an OW operator keep up". I
intuitively relate both problems to the learning capacity of a system to
adapt to the load, in order to perform both:
1. tear-down or warm-up action containers
2. shut-down or start-up new VMs to run actions


On Thu, May 31, 2018 at 10:36 AM Michele Sciabarra <michele@sciabarra.com>
wrote:

> The recent work introduced also support to use docker image as a compiler
> (and it is what I normally do).
>
> I "compile" the binary offliine and I send it as a single binary, that is
> started at init time.
>
> However while for Go and Swift it is "easy", I do not see how we can
> "precompile" a nodejs application.
> I checked around and it looks like NodeJS does a Just In Time Compilatoin,
> I have not found a way to cache the compilation.
>
> So this is where Go applications may have an edge...
>
> --
>   Michele Sciabarra
>   michele@sciabarra.com
>
> On Thu, May 31, 2018, at 1:34 PM, Nick Mitchell wrote:
> > for nodejs at least: the cost of a few requires of common packages can
> > easily get you up to the 150-200ms range (e.g. request is a big hitter;
> and
> > this is all on top of the cost of starting a container!). perhaps, for
> > nodejs at least, there are only a few options, ultimately: user pays more
> > for idle resources; provider pays more for idle stem cells; or users
> take a
> > very hard line on the modules they import.
> >
> > switching to other (compiled) runtimes might help, e.g. with the recent
> > work on precompiled go and swift actions? we'd still be left with the
> > container start times, but at least this is something we can control,
> e.g.
> > by requiring users to pay more for access to a larger prewarmed pool?
> >
> > nick
> >
> >
> > On Thu, May 31, 2018 at 7:22 AM, James Thomas <jthomas.uk@gmail.com>
> wrote:
> >
> > > One of most frequent complaints[1][2][3] I hear from developers using
> > > serverless platforms is coping with cold-start latency when dealing
> with
> > > sudden bursts of traffic.
> > >
> > > Developers often ask for a feature where they can set the number of
> warm
> > > containers kept in the cache for a function. This would allow them to
> keep
> > > a higher number of warm containers for applications with bursty traffic
> > > and/or upgrade the cached number prior to an anticpated burst of
> traffic
> > > arriving. This would be exposed by the managed platforms as a chargable
> > > feature.
> > >
> > > Is this something we could support on OpenWhisk? Ignoring the
> complexity
> > > and feasibility of any solution, from a developer POV I can image
> having an
> > > action annotation `max-warm` which would set the maximum number of warm
> > > containers allowed in the cache.
> > >
> > > Tyson is currently working on concurrent activation processing, which
> is
> > > one approach to reducing cold-start delays[4]. However, there are some
> > > downsides to concurrent activations, like no runtime isolation for
> request
> > > processing, which might make this feature inappropraite for some users.
> > >
> > > [1]
> > > https://www.reddit.com/r/aws/comments/6w1hip/how_many_
> > > successive_lambda_invocations_will_use_a/
> > > [2]
> > > https://twitter.com/search?f=tweets&vertical=default&q=%20%
> > > 23AWSWishlist%20warm&src=typd
> > > [3]
> > > https://theburningmonk.com/2018/01/im-afraid-youre-
> > > thinking-about-aws-lambda-cold-starts-all-wrong/
> > > [4] - https://github.com/apache/incubator-openwhisk/pull/2795
> > >
> > > --
> > > Regards,
> > > James Thomas
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message