Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id CD984200D08 for ; Thu, 7 Sep 2017 04:15:39 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id CC1571609DA; Thu, 7 Sep 2017 02:15:39 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 20A611609C5 for ; Thu, 7 Sep 2017 04:15:38 +0200 (CEST) Received: (qmail 8981 invoked by uid 500); 7 Sep 2017 02:15:38 -0000 Mailing-List: contact commits-help@kudu.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kudu.apache.org Delivered-To: mailing list commits@kudu.apache.org Received: (qmail 8972 invoked by uid 99); 7 Sep 2017 02:15:38 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Sep 2017 02:15:38 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 1D87AF5533; Thu, 7 Sep 2017 02:15:38 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: adar@apache.org To: commits@kudu.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: kudu git commit: KUDU-2083. Decrement running maintenance ops on failed prepare Date: Thu, 7 Sep 2017 02:15:38 +0000 (UTC) archived-at: Thu, 07 Sep 2017 02:15:40 -0000 Repository: kudu Updated Branches: refs/heads/branch-1.2.x 62b96231b -> 7b6755d3a KUDU-2083. Decrement running maintenance ops on failed prepare There is currently a bug where we don't decrement the number of running ops when an op->Prepare() fails. Although rare, when this bug is hit, it will decrease the number of simultaneous mm ops that can run until none can, causing the tserver to run OOM. Change-Id: I8022bcd4c6470dfef2dece0cbefede916a752291 Reviewed-on: http://gerrit.cloudera.org:8080/7610 Reviewed-by: Todd Lipcon Tested-by: Kudu Jenkins (cherry picked from commit b365ad0bd6372d459f547da0f1cb82e36148c541) Reviewed-on: http://gerrit.cloudera.org:8080/7970 Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/7b6755d3 Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/7b6755d3 Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/7b6755d3 Branch: refs/heads/branch-1.2.x Commit: 7b6755d3a2ceae01e19f163d155304a33b2b9368 Parents: 62b9623 Author: David Alves Authored: Mon Aug 7 15:42:34 2017 -0700 Committer: Todd Lipcon Committed: Wed Sep 6 17:37:55 2017 +0000 ---------------------------------------------------------------------- src/kudu/util/maintenance_manager.cc | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kudu/blob/7b6755d3/src/kudu/util/maintenance_manager.cc ---------------------------------------------------------------------- diff --git a/src/kudu/util/maintenance_manager.cc b/src/kudu/util/maintenance_manager.cc index 6a50f6f..e56d10b 100644 --- a/src/kudu/util/maintenance_manager.cc +++ b/src/kudu/util/maintenance_manager.cc @@ -227,6 +227,7 @@ void MaintenanceManager::RunSchedulerThread() { LOG(INFO) << "Prepare failed for " << op->name() << ". Re-running scheduler."; op->running_--; + running_ops_--; op->cond_->Signal(); continue; }