Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9F8A2200B43 for ; Wed, 29 Jun 2016 03:20:26 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9E33B160A6E; Wed, 29 Jun 2016 01:20:26 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EFC85160A6F for ; Wed, 29 Jun 2016 03:20:25 +0200 (CEST) Received: (qmail 69005 invoked by uid 500); 29 Jun 2016 01:20:25 -0000 Mailing-List: contact commits-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list commits@hawq.incubator.apache.org Received: (qmail 68995 invoked by uid 99); 29 Jun 2016 01:20:25 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Jun 2016 01:20:25 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 9BB72C030A for ; Wed, 29 Jun 2016 01:20:24 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 2Ji3fVE4OfKm for ; Wed, 29 Jun 2016 01:20:22 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id BE95A5FCF7 for ; Wed, 29 Jun 2016 01:20:21 +0000 (UTC) Received: (qmail 68493 invoked by uid 99); 29 Jun 2016 01:20:21 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Jun 2016 01:20:21 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id D9499DFDA9; Wed, 29 Jun 2016 01:20:20 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: cjcjameson@apache.org To: commits@hawq.incubator.apache.org Date: Wed, 29 Jun 2016 01:21:23 -0000 Message-Id: <9ed509e6cc9d4caf8044984c2094bb76@git.apache.org> In-Reply-To: <2cb4ce1ff5994c95911c10b8de4ba8e9@git.apache.org> References: <2cb4ce1ff5994c95911c10b8de4ba8e9@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [65/80] [abbrv] incubator-hawq git commit: HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down archived-at: Wed, 29 Jun 2016 01:20:26 -0000 HAWQ-855. Make resource manager ask for cluster report quickly when YARN is detected down Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/48a79cc2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/48a79cc2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/48a79cc2 Branch: refs/heads/HAWQ-537 Commit: 48a79cc270bb8817d9f58b64fd5abb61f8ef0643 Parents: 2c87cdc Author: YI JIN Authored: Thu Jun 23 13:31:32 2016 +1000 Committer: YI JIN Committed: Thu Jun 23 13:31:32 2016 +1000 ---------------------------------------------------------------------- src/backend/resourcemanager/include/resourcepool.h | 2 +- src/backend/resourcemanager/resourcemanager.c | 2 +- src/backend/resourcemanager/resourcepool.c | 6 +++++- 3 files changed, 7 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/48a79cc2/src/backend/resourcemanager/include/resourcepool.h ---------------------------------------------------------------------- diff --git a/src/backend/resourcemanager/include/resourcepool.h b/src/backend/resourcemanager/include/resourcepool.h index 4caf139..f23e77b 100644 --- a/src/backend/resourcemanager/include/resourcepool.h +++ b/src/backend/resourcemanager/include/resourcepool.h @@ -617,7 +617,7 @@ void timeoutIdleGRMResourceToRB(void); void forceReturnGRMResourceToRB(void); /* Check if some hosts are not updated from GRM cluster report. */ -bool hasSegmentGRMCapacityNotUpdated(void); +bool requireInstantClusterReport(void); bool allSegmentHasNoGRMContainersAllocated(void); int addOrderedResourceAllocTreeIndexByRatio(uint32_t ratio, BBST *tree); http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/48a79cc2/src/backend/resourcemanager/resourcemanager.c ---------------------------------------------------------------------- diff --git a/src/backend/resourcemanager/resourcemanager.c b/src/backend/resourcemanager/resourcemanager.c index 6e9e365..e511b97 100644 --- a/src/backend/resourcemanager/resourcemanager.c +++ b/src/backend/resourcemanager/resourcemanager.c @@ -2882,7 +2882,7 @@ void processResourceBrokerTasks(void) if ( (PRESPOOL->Segments.NodeCount > 0 ) && (curtime - PRESPOOL->LastUpdateTime > rm_cluster_report_period * 1000000LL || - hasSegmentGRMCapacityNotUpdated() ) && + requireInstantClusterReport() ) && (curtime - PRESPOOL->LastRequestTime > 5LL * 1000000LL) ) { double maxcap = 0.0; http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/48a79cc2/src/backend/resourcemanager/resourcepool.c ---------------------------------------------------------------------- diff --git a/src/backend/resourcemanager/resourcepool.c b/src/backend/resourcemanager/resourcepool.c index dc48e70..567b295 100644 --- a/src/backend/resourcemanager/resourcepool.c +++ b/src/backend/resourcemanager/resourcepool.c @@ -3711,7 +3711,7 @@ void timeoutIdleGRMResourceToRBByRatio(int ratioindex, validateResourcePoolStatus(false); } -bool hasSegmentGRMCapacityNotUpdated(void) +bool requireInstantClusterReport(void) { if (DRMGlobalInstance->ImpType == NONE_HAWQ2) { @@ -3741,6 +3741,10 @@ bool hasSegmentGRMCapacityNotUpdated(void) res = true; break; } + else if ( segresource->Stat->StatusDesc & SEG_STATUS_NO_GRM_NODE_REPORT ) { + res = true; + break; + } } freePAIRRefList(&(PRESPOOL->Segments), &allsegres); return res;