Mailing-List: contact issues-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@aurora.apache.org
Date: Tue, 14 Feb 2017 21:11:42 +0000 (UTC)
From: "David McLaughlin (JIRA)" <jira@apache.org>
To: issues@aurora.apache.org
Message-ID: <JIRA.13043072.1487106645000.80179.1487106702038@Atlassian.JIRA>
In-Reply-To: <JIRA.13043072.1487106645000@Atlassian.JIRA>
References: <JIRA.13043072.1487106645000@Atlassian.JIRA> <JIRA.13043072.1487106645018@jira-lw-us.apache.org>
Subject: [jira] [Created] (AURORA-1893) Add ability to prune tasks using
 aurora_admin
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 14 Feb 2017 21:12:16 -0000

David McLaughlin created AURORA-1893:
----------------------------------------

             Summary: Add ability to prune tasks using aurora_admin
                 Key: AURORA-1893
                 URL: https://issues.apache.org/jira/browse/AURORA-1893
             Project: Aurora
          Issue Type: Task
            Reporter: David McLaughlin


Currently task history pruning is configured with three main settings: maximum terminal tasks per job, maximum time to retain terminal tasks and a minimum time to retain terminal tasks.

There are times where a combination of bad actors in the cluster and the minimum time to retain terminal tasks can lead to incredibly bloated task store sizes, leading to serious problems with GC pressure during task store queries, and also when creating and persisting snapshots. 

At Twitter we've run into this and have had to respond by redeploying the Scheduler with more aggressive task pruning settings - which affects every user in the cluster. 

What we'd like is an endpoint in aurora_admin that accepts a TaskQuery and will prune all inactive tasks that match. This should allow us to limit the pruning by role, environment and also limit the number of tasks pruned. 


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)