aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Morozov <igm...@gmail.com>
Subject Finalization wait timeout in thermos executor for a task's teardown sequence
Date Thu, 19 May 2016 00:46:54 GMT
Folks,

We need to support a use case here at Uber when service processes that
don't respect SIGTERM signal and get killed after a default hardcoded
preemption timeout of 1 minute during task kill or task restart. That
significantly slows down upgrade workflow for such services.
We'd like to control this timeout, essentially reducing it to 5-10 seconds.

My current thinking is to expose preemption_wait timeout

class ThermosTaskRunner(TaskRunner):
....
THERMOS_PREEMPTION_WAIT = Amount(1, Time.MINUTES)

in thermos executor flags and set it in
DefaultThermosTaskRunnerProvider eventually propagating to all
ThermosRunner tasks.

A proper fix would be probably something in the line of making this
timeout configurable per task config but that would involve changing
pystachio thermos schema.

Thoughts?

-Igor Morozov

Mime
View raw message