Mailing-List: contact dev-help@openwhisk.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@openwhisk.apache.org
MIME-Version: 1.0
From: Jeremias Werner <jeremias.werner@gmail.com>
Date: Wed, 8 Feb 2017 14:27:50 +0100
Message-ID: <CAPBNrtGpWxucppEkvRGAsaKRQ=CqarRKi9R2sBwLU=Esoz+z+w@mail.gmail.com>
Subject: Exposing garbage collector for nodejs runtime
To: dev@openwhisk.apache.org
Content-Type: multipart/alternative; boundary=f403045c6482c7129b054804d498
archived-at: Wed, 08 Feb 2017 13:27:58 -0000

--f403045c6482c7129b054804d498
Content-Type: text/plain; charset=UTF-8

Hi,

I would like to get opinions and ideas on a finding and proposed solution
for memory intensive nodejs actions that I found during debugging of a
related issue.

*Problem Statement*
The problem is that the node js problem does not seem to have enough idle
time in a serverless world to run garbage collector. That can be seen by
running a 512MB action that allocates and free 128MB memory. As a result
you can see that every 5th invocation run into an application developer
error because OW's DOS protection kills the container at the 512MB limit.
(disclaimer: the action is a very synthetic action doing nothing else which
might reduce the time for the GC to jump in, like downloading a
video/image, ...)

You can see the details in the following PR:
https://github.com/openwhisk/openwhisk/pull/1826

*Implemented MVP in the PR:*
Run the node js process by default with "--expose-gc" and give the user the
chance to force garbage collection in memory intensive actions. Document
and/or blog about the use case.

*Advanced Proposal 1*
We could run the garbage collector anytime after the user code has been
executed in our nodejs skeleton app. This would result in a constant 30ms
overhead on top of the users action (see experiment in PR)

*Advanced Proposal 2*
Automatically detect memory intensive actions by gathering the memory
consumption on the node process and force the GC if necessary

*Advanced Proposal 3*
Provide a flag in the create action API to define whether the GC should be
enforced for the action.

Feedback / comments welcome!

kind regards,

Jeremias Werner

--f403045c6482c7129b054804d498--