Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id CB0E8200C17 for ; Fri, 10 Feb 2017 17:29:45 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id C8991160B5C; Fri, 10 Feb 2017 16:29:45 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1C91E160B4E for ; Fri, 10 Feb 2017 17:29:44 +0100 (CET) Received: (qmail 74270 invoked by uid 500); 10 Feb 2017 16:29:44 -0000 Mailing-List: contact issues-help@aurora.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@aurora.apache.org Delivered-To: mailing list issues@aurora.apache.org Received: (qmail 74261 invoked by uid 99); 10 Feb 2017 16:29:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Feb 2017 16:29:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id DC411C12D1 for ; Fri, 10 Feb 2017 16:29:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id G01ASCaIqLbQ for ; Fri, 10 Feb 2017 16:29:43 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id BBBD85F645 for ; Fri, 10 Feb 2017 16:29:42 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 01F96E0156 for ; Fri, 10 Feb 2017 16:29:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id B3F2A21D65 for ; Fri, 10 Feb 2017 16:29:41 +0000 (UTC) Date: Fri, 10 Feb 2017 16:29:41 +0000 (UTC) From: "Mehrdad Nurolahzade (JIRA)" To: issues@aurora.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (AURORA-1837) Improve task history pruning MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 10 Feb 2017 16:29:46 -0000 [ https://issues.apache.org/jira/browse/AURORA-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehrdad Nurolahzade updated AURORA-1837: ---------------------------------------- Summary: Improve task history pruning (was: Adding delay on pruning inactive jobs) > Improve task history pruning > ---------------------------- > > Key: AURORA-1837 > URL: https://issues.apache.org/jira/browse/AURORA-1837 > Project: Aurora > Issue Type: Task > Reporter: Reza Motamedi > Priority: Minor > Labels: scheduler > > TaskHistoryPrunner registers all inactive tasks upon _state_ change for pruning. > TaskHistoryPrunner::registerInactiveTask uses delay executor to schedule the process of prunning _task_s and _job_s. This is totally reasonable since pruning in not critical and can be done when the load on the scheduler is low. > Once pruning tasks, a delay is used in the first pruning phase (shutdownOnError) but in the second one seems to be instant. This has caused problems when lots of tasks are changing state and the load on the scheduler is high (for instance during scheduler restore). > to do items: > 1. investigate if we can add a delay to all executions, and what the delays should be. > 2. investigate if executions can be suppressed based on the load on the scheduler. -- This message was sent by Atlassian JIRA (v6.3.15#6346)