Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3B97A200C08 for ; Thu, 12 Jan 2017 04:10:25 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 3A0BB160B50; Thu, 12 Jan 2017 03:10:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 39C8E160B4E for ; Thu, 12 Jan 2017 04:10:24 +0100 (CET) Received: (qmail 38436 invoked by uid 500); 12 Jan 2017 03:10:23 -0000 Mailing-List: contact commits-help@quickstep.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@quickstep.incubator.apache.org Delivered-To: mailing list commits@quickstep.incubator.apache.org Received: (qmail 38426 invoked by uid 99); 12 Jan 2017 03:10:23 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jan 2017 03:10:23 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E819E1A001B for ; Thu, 12 Jan 2017 03:10:22 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.219 X-Spam-Level: X-Spam-Status: No, score=-6.219 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id Gf0WZrB23v6T for ; Thu, 12 Jan 2017 03:10:20 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id A77725FB54 for ; Thu, 12 Jan 2017 03:10:19 +0000 (UTC) Received: (qmail 38406 invoked by uid 99); 12 Jan 2017 03:10:18 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jan 2017 03:10:18 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id C3D50DFAF2; Thu, 12 Jan 2017 03:10:18 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: zuyuz@apache.org To: commits@quickstep.incubator.apache.org Message-Id: <3c65e8a63a524f0db323b0a4443979e8@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: incubator-quickstep git commit: Refactored SelectOperator::getAllWorkOrders. [Forced Update!] Date: Thu, 12 Jan 2017 03:10:18 +0000 (UTC) archived-at: Thu, 12 Jan 2017 03:10:25 -0000 Repository: incubator-quickstep Updated Branches: refs/heads/refactor-select 4dff05a4e -> 86e65d032 (forced update) Refactored SelectOperator::getAllWorkOrders. Project: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/commit/86e65d03 Tree: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/tree/86e65d03 Diff: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/diff/86e65d03 Branch: refs/heads/refactor-select Commit: 86e65d032999bd5269d60d5d463b696c57d36134 Parents: 8d25d43 Author: Zuyu Zhang Authored: Wed Jan 11 18:27:59 2017 -0800 Committer: Zuyu Zhang Committed: Wed Jan 11 19:10:13 2017 -0800 ---------------------------------------------------------------------- relational_operators/SelectOperator.cpp | 192 +++++++++------------------ relational_operators/SelectOperator.hpp | 14 -- 2 files changed, 61 insertions(+), 145 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/86e65d03/relational_operators/SelectOperator.cpp ---------------------------------------------------------------------- diff --git a/relational_operators/SelectOperator.cpp b/relational_operators/SelectOperator.cpp index 236ee7c..f5c9ee9 100644 --- a/relational_operators/SelectOperator.cpp +++ b/relational_operators/SelectOperator.cpp @@ -22,6 +22,7 @@ #include #include +#include "catalog/CatalogTypedefs.hpp" #include "query_execution/QueryContext.hpp" #include "query_execution/WorkOrderProtosContainer.hpp" #include "query_execution/WorkOrdersContainer.hpp" @@ -43,104 +44,6 @@ namespace quickstep { class Predicate; -void SelectOperator::addWorkOrders(WorkOrdersContainer *container, - QueryContext *query_context, - StorageManager *storage_manager, - const Predicate *predicate, - const std::vector> *selection, - InsertDestination *output_destination) { - if (input_relation_is_stored_) { - for (const block_id input_block_id : input_relation_block_ids_) { - container->addNormalWorkOrder( - new SelectWorkOrder( - query_id_, - input_relation_, - input_block_id, - predicate, - simple_projection_, - simple_selection_, - selection, - output_destination, - storage_manager, - CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context)), - op_index_); - } - } else { - while (num_workorders_generated_ < input_relation_block_ids_.size()) { - container->addNormalWorkOrder( - new SelectWorkOrder( - query_id_, - input_relation_, - input_relation_block_ids_[num_workorders_generated_], - predicate, - simple_projection_, - simple_selection_, - selection, - output_destination, - storage_manager, - CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context)), - op_index_); - ++num_workorders_generated_; - } - } -} - -#ifdef QUICKSTEP_HAVE_LIBNUMA -void SelectOperator::addPartitionAwareWorkOrders(WorkOrdersContainer *container, - QueryContext *query_context, - StorageManager *storage_manager, - const Predicate *predicate, - const std::vector> *selection, - InsertDestination *output_destination) { - DCHECK(placement_scheme_ != nullptr); - const std::size_t num_partitions = input_relation_.getPartitionScheme().getPartitionSchemeHeader().getNumPartitions(); - if (input_relation_is_stored_) { - for (std::size_t part_id = 0; part_id < num_partitions; ++part_id) { - for (const block_id input_block_id : - input_relation_block_ids_in_partition_[part_id]) { - container->addNormalWorkOrder( - new SelectWorkOrder( - query_id_, - input_relation_, - input_block_id, - predicate, - simple_projection_, - simple_selection_, - selection, - output_destination, - storage_manager, - CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context), - placement_scheme_->getNUMANodeForBlock(input_block_id)), - op_index_); - } - } - } else { - for (std::size_t part_id = 0; part_id < num_partitions; ++part_id) { - while (num_workorders_generated_in_partition_[part_id] < - input_relation_block_ids_in_partition_[part_id].size()) { - block_id block_in_partition - = input_relation_block_ids_in_partition_[part_id][num_workorders_generated_in_partition_[part_id]]; - container->addNormalWorkOrder( - new SelectWorkOrder( - query_id_, - input_relation_, - block_in_partition, - predicate, - simple_projection_, - simple_selection_, - selection, - output_destination, - storage_manager, - CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context), - placement_scheme_->getNUMANodeForBlock(block_in_partition)), - op_index_); - ++num_workorders_generated_in_partition_[part_id]; - } - } - } -} -#endif - bool SelectOperator::getAllWorkOrders( WorkOrdersContainer *container, QueryContext *query_context, @@ -159,48 +62,75 @@ bool SelectOperator::getAllWorkOrders( query_context->getInsertDestination(output_destination_index_); if (input_relation_is_stored_) { - if (!started_) { - if (input_relation_.hasPartitionScheme()) { + if (started_) { + return true; + } + + if (input_relation_.hasPartitionScheme()) { + const std::size_t num_partitions = + input_relation_.getPartitionScheme().getPartitionSchemeHeader().getNumPartitions(); + + for (std::size_t part_id = 0; part_id < num_partitions; ++part_id) { + for (const block_id input_block_id : input_relation_block_ids_in_partition_[part_id]) { + numa_node_id numa_node = 0; #ifdef QUICKSTEP_HAVE_LIBNUMA - if (input_relation_.hasNUMAPlacementScheme()) { - addPartitionAwareWorkOrders(container, - query_context, - storage_manager, - predicate, - selection, - output_destination); + if (input_relation_.hasNUMAPlacementScheme()) { + numa_node = placement_scheme_->getNUMANodeForBlock(input_block_id); + } +#endif // QUICKSTEP_HAVE_LIBNUMA + container->addNormalWorkOrder( + new SelectWorkOrder(query_id_, input_relation_, input_block_id, predicate, simple_projection_, + simple_selection_, selection, output_destination, storage_manager, + CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context), numa_node), + op_index_); } -#endif - } else { - addWorkOrders(container, - query_context, - storage_manager, - predicate, - selection, - output_destination); } - started_ = true; + } else { + for (const block_id input_block_id : input_relation_block_ids_) { + container->addNormalWorkOrder( + new SelectWorkOrder(query_id_, input_relation_, input_block_id, predicate, simple_projection_, + simple_selection_, selection, output_destination, storage_manager, + CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context)), + op_index_); + } } - return started_; + started_ = true; + return true; } else { if (input_relation_.hasPartitionScheme()) { + const std::size_t num_partitions = + input_relation_.getPartitionScheme().getPartitionSchemeHeader().getNumPartitions(); + + for (std::size_t part_id = 0; part_id < num_partitions; ++part_id) { + while (num_workorders_generated_in_partition_[part_id] < + input_relation_block_ids_in_partition_[part_id].size()) { + const block_id block_in_partition + = input_relation_block_ids_in_partition_[part_id][num_workorders_generated_in_partition_[part_id]]; + + numa_node_id numa_node = 0; #ifdef QUICKSTEP_HAVE_LIBNUMA - if (input_relation_.hasNUMAPlacementScheme()) { - addPartitionAwareWorkOrders(container, - query_context, - storage_manager, - predicate, - selection, - output_destination); + if (input_relation_.hasNUMAPlacementScheme()) { + numa_node = placement_scheme_->getNUMANodeForBlock(block_in_partition); + } +#endif // QUICKSTEP_HAVE_LIBNUMA + container->addNormalWorkOrder( + new SelectWorkOrder(query_id_, input_relation_, block_in_partition, predicate, simple_projection_, + simple_selection_, selection, output_destination, storage_manager, + CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context), numa_node), + op_index_); + ++num_workorders_generated_in_partition_[part_id]; } -#endif + } } else { - addWorkOrders(container, - query_context, - storage_manager, - predicate, - selection, - output_destination); + while (num_workorders_generated_ < input_relation_block_ids_.size()) { + container->addNormalWorkOrder( + new SelectWorkOrder(query_id_, input_relation_, input_relation_block_ids_[num_workorders_generated_], + predicate, simple_projection_, simple_selection_, selection, output_destination, + storage_manager, + CreateLIPFilterAdaptiveProberHelper(lip_deployment_index_, query_context)), + op_index_); + ++num_workorders_generated_; + } } return done_feeding_input_relation_; } http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/86e65d03/relational_operators/SelectOperator.hpp ---------------------------------------------------------------------- diff --git a/relational_operators/SelectOperator.hpp b/relational_operators/SelectOperator.hpp index 2ace458..6741f45 100644 --- a/relational_operators/SelectOperator.hpp +++ b/relational_operators/SelectOperator.hpp @@ -248,20 +248,6 @@ class SelectOperator : public RelationalOperator { return output_relation_.getID(); } - void addWorkOrders(WorkOrdersContainer *container, - QueryContext *query_context, - StorageManager *storage_manager, - const Predicate *predicate, - const std::vector> *selection, - InsertDestination *output_destination); - - void addPartitionAwareWorkOrders(WorkOrdersContainer *container, - QueryContext *query_context, - StorageManager *storage_manager, - const Predicate *predicate, - const std::vector> *selection, - InsertDestination *output_destination); - private: /** * @brief Create Work Order proto.