Return-Path: X-Original-To: apmail-hawq-commits-archive@minotaur.apache.org Delivered-To: apmail-hawq-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4129118869 for ; Fri, 26 Feb 2016 11:04:03 +0000 (UTC) Received: (qmail 84346 invoked by uid 500); 26 Feb 2016 11:04:03 -0000 Delivered-To: apmail-hawq-commits-archive@hawq.apache.org Received: (qmail 84310 invoked by uid 500); 26 Feb 2016 11:04:03 -0000 Mailing-List: contact commits-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list commits@hawq.incubator.apache.org Received: (qmail 84300 invoked by uid 99); 26 Feb 2016 11:04:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Feb 2016 11:04:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id BA243C28AA for ; Fri, 26 Feb 2016 11:04:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.221 X-Spam-Level: X-Spam-Status: No, score=-3.221 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id m1ZOwuN4ffJb for ; Fri, 26 Feb 2016 11:04:01 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 3D0FC5FAED for ; Fri, 26 Feb 2016 11:04:00 +0000 (UTC) Received: (qmail 84281 invoked by uid 99); 26 Feb 2016 11:03:59 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Feb 2016 11:03:59 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 56924DFBC9; Fri, 26 Feb 2016 11:03:59 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: huor@apache.org To: commits@hawq.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: incubator-hawq git commit: Revert "HAWQ-453. Do not allocate query resource in prepare stage for prepared statement" Date: Fri, 26 Feb 2016 11:03:59 +0000 (UTC) Repository: incubator-hawq Updated Branches: refs/heads/master 598bb4366 -> b8b9a0e84 Revert "HAWQ-453. Do not allocate query resource in prepare stage for prepared statement" This reverts commit 5069b7b0fdbb4de9230a63e4793fb55ef254f586. Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/b8b9a0e8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/b8b9a0e8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/b8b9a0e8 Branch: refs/heads/master Commit: b8b9a0e843dad2ac1e413d184979b463f5d5b4b2 Parents: 598bb43 Author: Ruilong Huo Authored: Fri Feb 26 18:59:23 2016 +0800 Committer: Ruilong Huo Committed: Fri Feb 26 18:59:23 2016 +0800 ---------------------------------------------------------------------- src/backend/cdb/cdbdatalocality.c | 58 +++---------------------- src/backend/executor/spi.c | 72 +++---------------------------- src/backend/optimizer/plan/planner.c | 3 +- src/include/cdb/cdbdatalocality.h | 2 +- src/include/executor/spi.h | 7 --- 5 files changed, 15 insertions(+), 127 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b8b9a0e8/src/backend/cdb/cdbdatalocality.c ---------------------------------------------------------------------- diff --git a/src/backend/cdb/cdbdatalocality.c b/src/backend/cdb/cdbdatalocality.c index 9a5bedb..0561d2e 100644 --- a/src/backend/cdb/cdbdatalocality.c +++ b/src/backend/cdb/cdbdatalocality.c @@ -41,7 +41,6 @@ #include "utils/tqual.h" #include "utils/memutils.h" #include "executor/execdesc.h" -#include "executor/spi.h" #include "nodes/nodes.h" #include "nodes/parsenodes.h" #include "optimizer/walkers.h" @@ -3960,11 +3959,10 @@ static void cleanup_allocation_algorithm( /* * calculate_planner_segment_num - * fixedVsegNum is used by PBE, since all the execute should use the same number of vsegs. */ SplitAllocResult * calculate_planner_segment_num(Query *query, QueryResourceLife resourceLife, - List *fullRangeTable, GpPolicy *intoPolicy, int sliceNum, int fixedVsegNum) { + List *fullRangeTable, GpPolicy *intoPolicy, int sliceNum) { SplitAllocResult *result = NULL; QueryResource *resource = NULL; QueryResourceParameters *resource_parameters = NULL; @@ -4048,15 +4046,7 @@ calculate_planner_segment_num(Query *query, QueryResourceLife resourceLife, /*use inherit resource*/ if (resourceLife == QRL_INHERIT) { - - if ( SPI_IsInPrepare() && (GetActiveQueryResource() == NULL) ) - { - resource = NULL; - } - else - { - resource = AllocateResource(resourceLife, sliceNum, 0, 0, 0, NULL, 0); - } + resource = AllocateResource(resourceLife, sliceNum, 0, 0, 0, NULL, 0); saveQueryResourceParameters( resource_parameters, /* resource_parameters */ @@ -4207,11 +4197,6 @@ calculate_planner_segment_num(Query *query, QueryResourceLife resourceLife, maxTargetSegmentNumber = enforce_virtual_segment_number; minTargetSegmentNumber = enforce_virtual_segment_number; } - /* in PBE mode, the execute should use the same vseg number. */ - if(fixedVsegNum > 0 ){ - maxTargetSegmentNumber = fixedVsegNum; - minTargetSegmentNumber = fixedVsegNum; - } uint64_t before_rm_allocate_resource = gettime_microsec(); /* cost is use by RM to balance workload between hosts. the cost is at least one block size*/ @@ -4219,40 +4204,9 @@ calculate_planner_segment_num(Query *query, QueryResourceLife resourceLife, mincost <<= 20; int64 queryCost = context.total_size < mincost ? mincost : context.total_size; if (QRL_NONE != resourceLife) { - - if (SPI_IsInPrepare()) - { - resource = NULL; - /* - * prepare need to get resource quota from RM - * and pass quota(planner_segments) to Orca or Planner to generate plan - * the following executes(in PBE) should reallocate the same number - * of resources. - */ - uint32 seg_num; - uint32 seg_num_min; - uint32 seg_memory_mb; - double seg_core; - - GetResourceQuota(maxTargetSegmentNumber, - minTargetSegmentNumber, - &seg_num, - &seg_num_min, - &seg_memory_mb, - &seg_core); - - planner_segments = seg_num; - minTargetSegmentNumber = planner_segments; - maxTargetSegmentNumber = planner_segments; - } - else - { - resource = AllocateResource(QRL_ONCE, sliceNum, queryCost, - maxTargetSegmentNumber, - minTargetSegmentNumber, - context.host_context.hostnameVolInfos, - context.host_context.size); - } + resource = AllocateResource(QRL_ONCE, sliceNum, queryCost, + maxTargetSegmentNumber, minTargetSegmentNumber, + context.host_context.hostnameVolInfos, context.host_context.size); saveQueryResourceParameters( resource_parameters, /* resource_parameters */ @@ -4282,7 +4236,7 @@ calculate_planner_segment_num(Query *query, QueryResourceLife resourceLife, if (resource == NULL) { result->resource = NULL; - result->resource_parameters = resource_parameters; + result->resource_parameters = NULL; result->alloc_results = NIL; result->relsType = NIL; result->planner_segments = planner_segments; http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b8b9a0e8/src/backend/executor/spi.c ---------------------------------------------------------------------- diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c index 8e7645c..07812aa 100644 --- a/src/backend/executor/spi.c +++ b/src/backend/executor/spi.c @@ -46,7 +46,6 @@ #include "cdb/memquota.h" #include "executor/nodeFunctionscan.h" #include "nodes/stack.h" -#include "cdb/cdbdatalocality.h" extern char *savedSeqServerHost; extern int savedSeqServerPort; @@ -75,7 +74,6 @@ static int _SPI_curid = -1; static PGconn *_QD_conn = NULL; /* To call back to the QD for SQL execution */ static char *_QD_currently_prepared_stmt = NULL; -static int SPI_prepare_depth = 0; static void _SPI_prepare_plan(const char *src, SPIPlanPtr plan); @@ -108,32 +106,6 @@ static bool _SPI_checktuples(void); /* =================== interface functions =================== */ -bool SPI_IsInPrepare(void) -{ - if (SPI_prepare_depth > 0) - { - return true; - } - else if (SPI_prepare_depth < 0) - { - elog(ERROR, "Invalid SPI_prepare_depth %d while getting SPI prepare depth", - SPI_prepare_depth); - } - - return false; -} - -void SPI_IncreasePrepareDepth(void) -{ - SPI_prepare_depth++; -} - -void SPI_DecreasePrepareDepth(void) -{ - SPI_prepare_depth--; -} - - int SPI_connect(void) { @@ -594,8 +566,6 @@ SPI_prepare(const char *src, int nargs, Oid *argtypes) _SPI_plan plan; _SPI_plan *result; - SPI_IncreasePrepareDepth(); - if (src == NULL || nargs < 0 || (nargs > 0 && argtypes == NULL)) { SPI_result = SPI_ERROR_ARGUMENT; @@ -621,13 +591,9 @@ SPI_prepare(const char *src, int nargs, Oid *argtypes) /* copy plan to procedure context */ result = _SPI_copy_plan(&plan, _SPI_CPLAN_PROCXT); - - SPI_DecreasePrepareDepth(); } PG_CATCH(); { - SPI_DecreasePrepareDepth(); - _SPI_end_call(true); PG_RE_THROW(); } @@ -1853,37 +1819,13 @@ _SPI_execute_plan(_SPI_plan * plan, Datum *Values, const char *Nulls, (stmt->resource == NULL) && (stmt->resource_parameters != NULL) ) { - SplitAllocResult *allocResult = NULL; - - /* If this is a parallel plan. */ - if (stmt->planTree->dispatch == DISPATCH_PARALLEL) - { - /* - * Now, we want to allocate resource. - */ - allocResult = calculate_planner_segment_num(queryTree, - stmt->resource_parameters->life, - stmt->rtable, - stmt->intoPolicy, - stmt->nMotionNodes + stmt->nInitPlans + 1, - stmt->resource_parameters->min_target_segment_num); - - Assert(allocResult); - - if(stmt->resource !=NULL) - { - pfree(stmt->resource); - } - stmt->resource = allocResult->resource; - if(stmt->scantable_splits !=NULL) - { - list_free_deep(stmt->scantable_splits); - } - stmt->scantable_splits = allocResult->alloc_results; - stmt->planner_segments = allocResult->planner_segments; - stmt->datalocalityInfo = allocResult->datalocalityInfo; - pfree(allocResult); - } + stmt->resource = AllocateResource(stmt->resource_parameters->life, + stmt->resource_parameters->slice_size, + stmt->resource_parameters->iobytes, + stmt->resource_parameters->max_target_segment_num, + stmt->resource_parameters->min_target_segment_num, + stmt->resource_parameters->vol_info, + stmt->resource_parameters->vol_info_size); } originalStmt->resource = NULL; http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b8b9a0e8/src/backend/optimizer/plan/planner.c ---------------------------------------------------------------------- diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index c98ba49..824558b 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -495,8 +495,7 @@ resource_negotiator(Query *parse, int cursorOptions, ParamListInfo boundParams, */ allocResult = calculate_planner_segment_num(my_parse, resourceLife, plannedstmt->rtable, plannedstmt->intoPolicy, - plannedstmt->nMotionNodes + plannedstmt->nInitPlans + 1, - -1); + plannedstmt->nMotionNodes + plannedstmt->nInitPlans + 1); Assert(allocResult); http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b8b9a0e8/src/include/cdb/cdbdatalocality.h ---------------------------------------------------------------------- diff --git a/src/include/cdb/cdbdatalocality.h b/src/include/cdb/cdbdatalocality.h index 0cb93d4..95f2bd5 100644 --- a/src/include/cdb/cdbdatalocality.h +++ b/src/include/cdb/cdbdatalocality.h @@ -86,7 +86,7 @@ void saveQueryResourceParameters( * we calculate the appropriate planner segment_num. */ SplitAllocResult * calculate_planner_segment_num(Query *query, QueryResourceLife resourceLife, - List *rtable, GpPolicy *intoPolicy, int sliceNum, int fixedVsegNum); + List *rtable, GpPolicy *intoPolicy, int sliceNum); FILE *fp; FILE *fpaoseg; http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b8b9a0e8/src/include/executor/spi.h ---------------------------------------------------------------------- diff --git a/src/include/executor/spi.h b/src/include/executor/spi.h index 7b4ba1b..867daf5 100644 --- a/src/include/executor/spi.h +++ b/src/include/executor/spi.h @@ -155,11 +155,4 @@ extern uint64 SPI_GetMemoryReservation(void); extern void SPI_ReserveMemory(uint64 mem_reserved); extern bool SPI_IsMemoryReserved(void); -/** - * Query resource related routines. - */ -extern bool SPI_IsInPrepare(void); -extern void SPI_IncreasePrepareCounter(void); -extern void SPI_DecreasePrepareCounter(void); - #endif /* SPI_H */