Return-Path: X-Original-To: apmail-hawq-commits-archive@minotaur.apache.org Delivered-To: apmail-hawq-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4632218F58 for ; Wed, 11 Nov 2015 03:21:57 +0000 (UTC) Received: (qmail 51217 invoked by uid 500); 11 Nov 2015 03:21:57 -0000 Delivered-To: apmail-hawq-commits-archive@hawq.apache.org Received: (qmail 51163 invoked by uid 500); 11 Nov 2015 03:21:57 -0000 Mailing-List: contact commits-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list commits@hawq.incubator.apache.org Received: (qmail 51154 invoked by uid 99); 11 Nov 2015 03:21:57 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Nov 2015 03:21:57 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id B69A5C80BD for ; Wed, 11 Nov 2015 03:21:56 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.436 X-Spam-Level: * X-Spam-Status: No, score=1.436 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.345, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id IzqfYRtYZfU6 for ; Wed, 11 Nov 2015 03:21:47 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with SMTP id A211E20B89 for ; Wed, 11 Nov 2015 03:21:46 +0000 (UTC) Received: (qmail 51067 invoked by uid 99); 11 Nov 2015 03:21:45 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Nov 2015 03:21:45 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id B2D95E048E; Wed, 11 Nov 2015 03:21:45 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: jian@apache.org To: commits@hawq.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: incubator-hawq git commit: HAWQ-117. Add query dispatching context checking before serialization Date: Wed, 11 Nov 2015 03:21:45 +0000 (UTC) Repository: incubator-hawq Updated Branches: refs/heads/master 5401b1045 -> 957309a24 HAWQ-117. Add query dispatching context checking before serialization Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/957309a2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/957309a2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/957309a2 Branch: refs/heads/master Commit: 957309a2459aa79bba98d973634fade0bc7c82db Parents: 5401b10 Author: Lirong Jian Authored: Thu Nov 5 14:10:38 2015 +0800 Committer: Lirong Jian Committed: Wed Nov 11 11:03:34 2015 +0800 ---------------------------------------------------------------------- src/backend/cdb/cdbcopy.c | 2 +- src/backend/cdb/cdbquerycontextdispatching.c | 12 ++++++++++-- src/backend/cdb/cdbsharedstorageop.c | 2 +- src/backend/commands/tablecmds.c | 6 ++++-- src/backend/executor/execMain.c | 2 +- src/backend/nodes/outfast.c | 5 +++++ src/backend/nodes/readfast.c | 2 ++ src/backend/tcop/postgres.c | 4 ++-- src/include/cdb/cdbquerycontextdispatching.h | 4 +++- 9 files changed, 29 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/cdb/cdbcopy.c ---------------------------------------------------------------------- diff --git a/src/backend/cdb/cdbcopy.c b/src/backend/cdb/cdbcopy.c index 81b55ec..8692e7b 100644 --- a/src/backend/cdb/cdbcopy.c +++ b/src/backend/cdb/cdbcopy.c @@ -207,7 +207,7 @@ cdbCopyStart(CdbCopy *c, char *copyCmd, Oid relid, Oid relerror, List *err_aoseg ((CopyStmt *)q->utilityStmt)->scantable_splits = scantable_splits; } - CloseQueryContextInfo(q->contextdisp); + FinalizeQueryContextInfo(q->contextdisp); /* * serialized the stmt tree, and dispatch it .... http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/cdb/cdbquerycontextdispatching.c ---------------------------------------------------------------------- diff --git a/src/backend/cdb/cdbquerycontextdispatching.c b/src/backend/cdb/cdbquerycontextdispatching.c index a1d44de..401e15d 100644 --- a/src/backend/cdb/cdbquerycontextdispatching.c +++ b/src/backend/cdb/cdbquerycontextdispatching.c @@ -223,8 +223,9 @@ CreateQueryContextInfo(void) AddAuxInfoToQueryContextInfo(retval); - return retval; + retval->finalized = false; + return retval; } /* @@ -251,9 +252,10 @@ InitQueryContextInfoFromFile(QueryContextInfo *cxt) * close a QueryContextInfo and close its file if it is opened. */ void -CloseQueryContextInfo(QueryContextInfo *cxt) +FinalizeQueryContextInfo(QueryContextInfo *cxt) { Assert(cxt); + Assert(!cxt->finalized); if (Debug_querycontext_print && Gp_role != GP_ROLE_EXECUTE) { @@ -304,6 +306,8 @@ CloseQueryContextInfo(QueryContextInfo *cxt) FileClose(cxt->file); cxt->file = 0; } + + cxt->finalized = true; } /* @@ -790,6 +794,7 @@ RebuildQueryContext(QueryContextInfo *cxt, HTAB **currentFilesystemCredentials, } } + cxt->finalized = false; /* * This is a bit overkill, but since we don't yet have a decent way to * determine individual cache entries affected by the dispatched @@ -808,6 +813,7 @@ void AddAuxInfoToQueryContextInfo(QueryContextInfo *cxt) { Assert(NULL != cxt); + Assert(!cxt->finalized); StringInfoData buffer; initStringInfo(&buffer); @@ -855,6 +861,7 @@ AddTupleToContextInfo(QueryContextInfo *cxt, Oid relid, const char *relname, HeapTuple tuple, int32 contentid) { Assert(NULL != cxt); + Assert(!cxt->finalized); StringInfoData header; initStringInfo(&header); @@ -990,6 +997,7 @@ AddTablespaceLocationToContextInfo(QueryContextInfo *cxt, Oid tspoid, const char *fmt) { Assert(NULL != cxt); + Assert(!cxt->finalized); StringInfoData header; initStringInfo(&header); http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/cdb/cdbsharedstorageop.c ---------------------------------------------------------------------- diff --git a/src/backend/cdb/cdbsharedstorageop.c b/src/backend/cdb/cdbsharedstorageop.c index 0eafe56..0f4c0f0 100644 --- a/src/backend/cdb/cdbsharedstorageop.c +++ b/src/backend/cdb/cdbsharedstorageop.c @@ -134,7 +134,7 @@ void PerformSharedStorageOpTasks(SharedStorageOpTasks *tasks, stat->numTasks = j; - CloseQueryContextInfo(contextdisp); + FinalizeQueryContextInfo(contextdisp); int gp_segments_for_planner_before = gp_segments_for_planner; http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/commands/tablecmds.c ---------------------------------------------------------------------- diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 626fad8..547725e 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -6446,7 +6446,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) OIDNewHeap, true, segment_segnos); - CloseQueryContextInfo(contextdisp); + FinalizeQueryContextInfo(contextdisp); ar_tab->newheap_oid = OIDNewHeap; @@ -6454,6 +6454,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) cdbdisp_iterate_results_sendback(result.result, result.numresults, UpdateCatalogModifiedOnSegments); dispatch_free_result(&result); + DropQueryContextInfo(contextdisp); FreeResource(resource); SetActiveQueryResource(savedResource); } @@ -16705,11 +16706,12 @@ ATPExecPartSplit(Relation rel, pc->scantable_splits = scantable_splits; pc->newpart_aosegnos = segment_segnos; + FinalizeQueryContextInfo(contextdisp); dispatch_statement_node((Node *) pc, contextdisp, resource, &result); cdbdisp_iterate_results_sendback(result.result, result.numresults, UpdateCatalogModifiedOnSegments); dispatch_free_result(&result); - CloseQueryContextInfo(contextdisp); + DropQueryContextInfo(contextdisp); FreeResource(resource); SetActiveQueryResource(savedResource); } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/executor/execMain.c ---------------------------------------------------------------------- diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 93b6893..f5a2038 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c @@ -887,7 +887,7 @@ ExecutorStart(QueryDesc *queryDesc, int eflags) cxt.info = plannedstmt->contextdisp; plan_tree_walker((Node *)plannedstmt->planTree, SetupSegnoForErrorTable, &cxt); - CloseQueryContextInfo(plannedstmt->contextdisp); + FinalizeQueryContextInfo(plannedstmt->contextdisp); } /* http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/nodes/outfast.c ---------------------------------------------------------------------- diff --git a/src/backend/nodes/outfast.c b/src/backend/nodes/outfast.c index a66896e..ec1cff9 100644 --- a/src/backend/nodes/outfast.c +++ b/src/backend/nodes/outfast.c @@ -3810,6 +3810,9 @@ _outAlterTypeStmt(StringInfo str, AlterTypeStmt *node) static void _outQueryContextInfo(StringInfo str, QueryContextInfo *node) { + /* Make sure this QueryContextInfo has been closed */ + Assert(node->finalized); + WRITE_NODE_TYPE("QUERYCONTEXTINFO"); WRITE_BOOL_FIELD(useFile); @@ -3822,6 +3825,8 @@ _outQueryContextInfo(StringInfo str, QueryContextInfo *node) WRITE_INT_FIELD(cursor); appendBinaryStringInfo(str, (const char *) node->buffer, node->cursor); } + + WRITE_BOOL_FIELD(finalized); } static void http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/nodes/readfast.c ---------------------------------------------------------------------- diff --git a/src/backend/nodes/readfast.c b/src/backend/nodes/readfast.c index 4a36f16..d61f1c6 100644 --- a/src/backend/nodes/readfast.c +++ b/src/backend/nodes/readfast.c @@ -4042,6 +4042,8 @@ _readQueryContextInfo(const char **str) *str += local_node->size; } + READ_BOOL_FIELD(finalized); + Assert(local_node->finalized); READ_DONE(); } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/backend/tcop/postgres.c ---------------------------------------------------------------------- diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index bcff3da..3f713c4 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -1020,7 +1020,7 @@ exec_mpp_query(const char *query_string, { RebuildQueryContext(query->contextdisp, ¤tFilesystemCredentials, ¤tFilesystemCredentialsMemoryContext); - CloseQueryContextInfo(query->contextdisp); + FinalizeQueryContextInfo(query->contextdisp); } } @@ -1072,7 +1072,7 @@ exec_mpp_query(const char *query_string, Assert(NULL != plan->contextdisp); RebuildQueryContext(plan->contextdisp, ¤tFilesystemCredentials, ¤tFilesystemCredentialsMemoryContext); - CloseQueryContextInfo(plan->contextdisp); + FinalizeQueryContextInfo(plan->contextdisp); } /* http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/957309a2/src/include/cdb/cdbquerycontextdispatching.h ---------------------------------------------------------------------- diff --git a/src/include/cdb/cdbquerycontextdispatching.h b/src/include/cdb/cdbquerycontextdispatching.h index 0a95f6d..f89a675 100644 --- a/src/include/cdb/cdbquerycontextdispatching.h +++ b/src/include/cdb/cdbquerycontextdispatching.h @@ -53,6 +53,8 @@ struct QueryContextInfo HTAB *htab; /* a hash table used to dedup */ List *errTblOid; /* already handled error table oid in the statement */ + + bool finalized; /* whether this query context info is closed */ }; typedef struct QueryContextInfo QueryContextInfo; @@ -102,7 +104,7 @@ extern void InitQueryContextInfoFromFile(QueryContextInfo *cxt); extern void -CloseQueryContextInfo(QueryContextInfo *cxt); +FinalizeQueryContextInfo(QueryContextInfo *cxt); extern void DropQueryContextInfo(QueryContextInfo *cxt);