aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mehrdad Nurolahzade (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AURORA-1837) Adding delay on pruning inactive jobs
Date Tue, 20 Dec 2016 21:06:58 GMT

    [ https://issues.apache.org/jira/browse/AURORA-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765247#comment-15765247
] 

Mehrdad Nurolahzade commented on AURORA-1837:
---------------------------------------------

One alternative, similar to job update history pruner, can be a two-step fixed-rate scheduled
task history pruner can identify and delete all to be deleted tasks since previous run in
step one and trim their associated jobs (only once) in step two. The initial delay can then
be tweaked to give scheduler enough time to recover before task history pruner kicks in.

> Adding delay on pruning inactive jobs
> -------------------------------------
>
>                 Key: AURORA-1837
>                 URL: https://issues.apache.org/jira/browse/AURORA-1837
>             Project: Aurora
>          Issue Type: Task
>            Reporter: Reza Motamedi
>            Priority: Minor
>              Labels: scheduler
>
> TaskHistoryPrunner registers all inactive tasks upon _state_ change for pruning. 
> TaskHistoryPrunner::registerInactiveTask uses delay executor to schedule the process
of prunning _task_s and _job_s. This is totally reasonable since pruning in not critical and
can be done when the load on the scheduler is low.
> Once pruning tasks, a delay is used in the first pruning phase (shutdownOnError) but
in the second one seems to be instant. This has caused problems when lots of tasks are changing
state and the load on the scheduler is high (for instance during scheduler restore).
> to do items:
> 1. investigate if we can add a delay to all executions, and what the delays should be.
> 2.  investigate if executions can be suppressed based on the load on the scheduler. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message