Return-Path: X-Original-To: apmail-quickstep-commits-archive@minotaur.apache.org Delivered-To: apmail-quickstep-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 615B4193E3 for ; Tue, 19 Apr 2016 16:44:21 +0000 (UTC) Received: (qmail 2247 invoked by uid 500); 19 Apr 2016 16:44:21 -0000 Delivered-To: apmail-quickstep-commits-archive@quickstep.apache.org Received: (qmail 2222 invoked by uid 500); 19 Apr 2016 16:44:21 -0000 Mailing-List: contact commits-help@quickstep.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@quickstep.incubator.apache.org Delivered-To: mailing list commits@quickstep.incubator.apache.org Received: (qmail 2213 invoked by uid 99); 19 Apr 2016 16:44:21 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Apr 2016 16:44:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id DFEBBC28BA for ; Tue, 19 Apr 2016 16:44:20 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.216 X-Spam-Level: X-Spam-Status: No, score=-4.216 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.996] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id Q1fzJgFYJ-Tj for ; Tue, 19 Apr 2016 16:44:18 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with SMTP id D2F9B5FACD for ; Tue, 19 Apr 2016 16:44:15 +0000 (UTC) Received: (qmail 1560 invoked by uid 99); 19 Apr 2016 16:44:15 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Apr 2016 16:44:15 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 054ACE020D; Tue, 19 Apr 2016 16:44:15 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: zuyuz@apache.org To: commits@quickstep.incubator.apache.org Date: Tue, 19 Apr 2016 16:44:29 -0000 Message-Id: <48fea3868903423cba977b3f29cc4a4f@git.apache.org> In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [16/24] incubator-quickstep git commit: Added hook (not actual code) to BitWeaving. (#169) http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/parser/preprocessed/SqlParser_gen.hpp ---------------------------------------------------------------------- diff --git a/parser/preprocessed/SqlParser_gen.hpp b/parser/preprocessed/SqlParser_gen.hpp index 807a39b..e35664f 100644 --- a/parser/preprocessed/SqlParser_gen.hpp +++ b/parser/preprocessed/SqlParser_gen.hpp @@ -1,8 +1,8 @@ -/* A Bison parser, made by GNU Bison 3.0.4. */ +/* A Bison parser, made by GNU Bison 3.0.2. */ /* Bison interface for Yacc-like parsers in C - Copyright (C) 1984, 1989-1990, 2000-2015 Free Software Foundation, Inc. + Copyright (C) 1984, 1989-1990, 2000-2013 Free Software Foundation, Inc. This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -72,101 +72,102 @@ extern int quickstep_yydebug; TOKEN_ASC = 282, TOKEN_BIGINT = 283, TOKEN_BIT = 284, - TOKEN_BLOCKPROPERTIES = 285, - TOKEN_BLOCKSAMPLE = 286, - TOKEN_BLOOM_FILTER = 287, - TOKEN_CSB_TREE = 288, - TOKEN_BY = 289, - TOKEN_CASE = 290, - TOKEN_CHARACTER = 291, - TOKEN_CHECK = 292, - TOKEN_COLUMN = 293, - TOKEN_CONSTRAINT = 294, - TOKEN_COPY = 295, - TOKEN_CREATE = 296, - TOKEN_DATE = 297, - TOKEN_DATETIME = 298, - TOKEN_DECIMAL = 299, - TOKEN_DEFAULT = 300, - TOKEN_DELETE = 301, - TOKEN_DELIMITER = 302, - TOKEN_DESC = 303, - TOKEN_DISTINCT = 304, - TOKEN_DOUBLE = 305, - TOKEN_DROP = 306, - TOKEN_ELSE = 307, - TOKEN_END = 308, - TOKEN_ESCAPE_STRINGS = 309, - TOKEN_EXISTS = 310, - TOKEN_EXTRACT = 311, - TOKEN_FALSE = 312, - TOKEN_FIRST = 313, - TOKEN_FLOAT = 314, - TOKEN_FOREIGN = 315, - TOKEN_FROM = 316, - TOKEN_FULL = 317, - TOKEN_GROUP = 318, - TOKEN_HASH = 319, - TOKEN_HAVING = 320, - TOKEN_IN = 321, - TOKEN_INDEX = 322, - TOKEN_INNER = 323, - TOKEN_INSERT = 324, - TOKEN_INTEGER = 325, - TOKEN_INTERVAL = 326, - TOKEN_INTO = 327, - TOKEN_JOIN = 328, - TOKEN_KEY = 329, - TOKEN_LAST = 330, - TOKEN_LEFT = 331, - TOKEN_LIMIT = 332, - TOKEN_LONG = 333, - TOKEN_NULL = 334, - TOKEN_NULLS = 335, - TOKEN_OFF = 336, - TOKEN_ON = 337, - TOKEN_ORDER = 338, - TOKEN_OUTER = 339, - TOKEN_PARTITION = 340, - TOKEN_PARTITIONS = 341, - TOKEN_PERCENT = 342, - TOKEN_PRIMARY = 343, - TOKEN_QUIT = 344, - TOKEN_RANGE = 345, - TOKEN_REAL = 346, - TOKEN_REFERENCES = 347, - TOKEN_RIGHT = 348, - TOKEN_ROW_DELIMITER = 349, - TOKEN_SELECT = 350, - TOKEN_SET = 351, - TOKEN_SMA = 352, - TOKEN_SMALLINT = 353, - TOKEN_TABLE = 354, - TOKEN_THEN = 355, - TOKEN_TIME = 356, - TOKEN_TIMESTAMP = 357, - TOKEN_TRUE = 358, - TOKEN_TUPLESAMPLE = 359, - TOKEN_UNIQUE = 360, - TOKEN_UPDATE = 361, - TOKEN_USING = 362, - TOKEN_VALUES = 363, - TOKEN_VARCHAR = 364, - TOKEN_WHEN = 365, - TOKEN_WHERE = 366, - TOKEN_WITH = 367, - TOKEN_YEARMONTH = 368, - TOKEN_EOF = 369, - TOKEN_LEX_ERROR = 370 + TOKEN_BITWEAVING = 285, + TOKEN_BLOCKPROPERTIES = 286, + TOKEN_BLOCKSAMPLE = 287, + TOKEN_BLOOM_FILTER = 288, + TOKEN_CSB_TREE = 289, + TOKEN_BY = 290, + TOKEN_CASE = 291, + TOKEN_CHARACTER = 292, + TOKEN_CHECK = 293, + TOKEN_COLUMN = 294, + TOKEN_CONSTRAINT = 295, + TOKEN_COPY = 296, + TOKEN_CREATE = 297, + TOKEN_DATE = 298, + TOKEN_DATETIME = 299, + TOKEN_DECIMAL = 300, + TOKEN_DEFAULT = 301, + TOKEN_DELETE = 302, + TOKEN_DELIMITER = 303, + TOKEN_DESC = 304, + TOKEN_DISTINCT = 305, + TOKEN_DOUBLE = 306, + TOKEN_DROP = 307, + TOKEN_ELSE = 308, + TOKEN_END = 309, + TOKEN_ESCAPE_STRINGS = 310, + TOKEN_EXISTS = 311, + TOKEN_EXTRACT = 312, + TOKEN_FALSE = 313, + TOKEN_FIRST = 314, + TOKEN_FLOAT = 315, + TOKEN_FOREIGN = 316, + TOKEN_FROM = 317, + TOKEN_FULL = 318, + TOKEN_GROUP = 319, + TOKEN_HASH = 320, + TOKEN_HAVING = 321, + TOKEN_IN = 322, + TOKEN_INDEX = 323, + TOKEN_INNER = 324, + TOKEN_INSERT = 325, + TOKEN_INTEGER = 326, + TOKEN_INTERVAL = 327, + TOKEN_INTO = 328, + TOKEN_JOIN = 329, + TOKEN_KEY = 330, + TOKEN_LAST = 331, + TOKEN_LEFT = 332, + TOKEN_LIMIT = 333, + TOKEN_LONG = 334, + TOKEN_NULL = 335, + TOKEN_NULLS = 336, + TOKEN_OFF = 337, + TOKEN_ON = 338, + TOKEN_ORDER = 339, + TOKEN_OUTER = 340, + TOKEN_PARTITION = 341, + TOKEN_PARTITIONS = 342, + TOKEN_PERCENT = 343, + TOKEN_PRIMARY = 344, + TOKEN_QUIT = 345, + TOKEN_RANGE = 346, + TOKEN_REAL = 347, + TOKEN_REFERENCES = 348, + TOKEN_RIGHT = 349, + TOKEN_ROW_DELIMITER = 350, + TOKEN_SELECT = 351, + TOKEN_SET = 352, + TOKEN_SMA = 353, + TOKEN_SMALLINT = 354, + TOKEN_TABLE = 355, + TOKEN_THEN = 356, + TOKEN_TIME = 357, + TOKEN_TIMESTAMP = 358, + TOKEN_TRUE = 359, + TOKEN_TUPLESAMPLE = 360, + TOKEN_UNIQUE = 361, + TOKEN_UPDATE = 362, + TOKEN_USING = 363, + TOKEN_VALUES = 364, + TOKEN_VARCHAR = 365, + TOKEN_WHEN = 366, + TOKEN_WHERE = 367, + TOKEN_WITH = 368, + TOKEN_YEARMONTH = 369, + TOKEN_EOF = 370, + TOKEN_LEX_ERROR = 371 }; #endif /* Value type. */ #if ! defined YYSTYPE && ! defined YYSTYPE_IS_DECLARED - +typedef union YYSTYPE YYSTYPE; union YYSTYPE { -#line 117 "../SqlParser.ypp" /* yacc.c:1915 */ +#line 117 "../SqlParser.ypp" /* yacc.c:1909 */ quickstep::ParseString *string_value_; @@ -256,10 +257,8 @@ union YYSTYPE quickstep::PtrVector *with_list_; quickstep::ParseSubqueryTableReference *with_list_element_; -#line 260 "SqlParser_gen.hpp" /* yacc.c:1915 */ +#line 261 "SqlParser_gen.hpp" /* yacc.c:1909 */ }; - -typedef union YYSTYPE YYSTYPE; # define YYSTYPE_IS_TRIVIAL 1 # define YYSTYPE_IS_DECLARED 1 #endif http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/parser/tests/Index.test ---------------------------------------------------------------------- diff --git a/parser/tests/Index.test b/parser/tests/Index.test index b5184bd..605768b 100644 --- a/parser/tests/Index.test +++ b/parser/tests/Index.test @@ -57,3 +57,12 @@ CreateIndexStatement[index_name=bloomIndex,relation_name=test, CREATE INDEX csbIndex ON test USING CSBTREE -- CreateIndexStatement[index_name=csbIndex,relation_name=test,index_type=cs_b_tree] +== +# Bitweaving is supported on builds where it was included as a submodule. The +# parser should create code to change the catalog, but on Block creation, no +# index will be created. +CREATE INDEX bwIndex ON test(int_col) USING bitweaving; +-- +CreateIndexStatement[index_name=bwIndex,relation_name=test,index_type=bitweaving] ++-attribute_list= + +-AttributeReference[attribute_name=int_col] http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/query_optimizer/CMakeLists.txt ---------------------------------------------------------------------- diff --git a/query_optimizer/CMakeLists.txt b/query_optimizer/CMakeLists.txt index 60319d4..44c8750 100644 --- a/query_optimizer/CMakeLists.txt +++ b/query_optimizer/CMakeLists.txt @@ -118,6 +118,7 @@ target_link_libraries(quickstep_queryoptimizer_ExecutionGenerator quickstep_storage_InsertDestination_proto quickstep_storage_StorageBlockLayout quickstep_storage_StorageBlockLayout_proto + quickstep_storage_SubBlockTypeRegistry quickstep_types_Type quickstep_types_Type_proto quickstep_types_TypedValue http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/query_optimizer/ExecutionGenerator.cpp ---------------------------------------------------------------------- diff --git a/query_optimizer/ExecutionGenerator.cpp b/query_optimizer/ExecutionGenerator.cpp index 43825b9..38ab09e 100644 --- a/query_optimizer/ExecutionGenerator.cpp +++ b/query_optimizer/ExecutionGenerator.cpp @@ -99,6 +99,7 @@ #include "storage/InsertDestination.pb.h" #include "storage/StorageBlockLayout.hpp" #include "storage/StorageBlockLayout.pb.h" +#include "storage/SubBlockTypeRegistry.hpp" #include "types/Type.hpp" #include "types/Type.pb.h" #include "types/TypedValue.hpp" @@ -870,7 +871,7 @@ void ExecutionGenerator::convertCreateIndex( THROW_SQL_ERROR() << "The relation " << input_relation->getName() << " already defines this index on the given attribute(s)."; } - if (!index_description.IsInitialized()) { + if (!SubBlockTypeRegistry::IndexDescriptionIsValid(*input_relation, index_description)) { // Check if the given index description is valid. THROW_SQL_ERROR() << "The index with given properties cannot be created."; } http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/CMakeLists.txt ---------------------------------------------------------------------- diff --git a/storage/CMakeLists.txt b/storage/CMakeLists.txt index 37cfc3d..11c2819 100644 --- a/storage/CMakeLists.txt +++ b/storage/CMakeLists.txt @@ -138,6 +138,19 @@ add_library(quickstep_storage_BasicColumnStoreValueAccessor ../empty_src.cpp BasicColumnStoreValueAccessor.hpp) add_library(quickstep_storage_BloomFilterIndexSubBlock BloomFilterIndexSubBlock.cpp BloomFilterIndexSubBlock.hpp) +# CMAKE_VALIDATE_IGNORE_BEGIN +if(QUICKSTEP_HAVE_BITWEAVING) + add_library(quickstep_storage_BitWeavingIndexSubBlock + bitweaving/BitWeavingIndexSubBlock.cpp + bitweaving/BitWeavingIndexSubBlock.hpp) + add_library(quickstep_storage_BitWeavingHIndexSubBlock + bitweaving/BitWeavingHIndexSubBlock.cpp + bitweaving/BitWeavingHIndexSubBlock.hpp) + add_library(quickstep_storage_BitWeavingVIndexSubBlock + bitweaving/BitWeavingVIndexSubBlock.cpp + bitweaving/BitWeavingVIndexSubBlock.hpp) +endif() +# CMAKE_VALIDATE_IGNORE_END add_library(quickstep_storage_ColumnStoreUtil ColumnStoreUtil.cpp ColumnStoreUtil.hpp) add_library(quickstep_storage_CompressedBlockBuilder CompressedBlockBuilder.cpp CompressedBlockBuilder.hpp) add_library(quickstep_storage_CompressedColumnStoreTupleStorageSubBlock @@ -316,6 +329,74 @@ target_link_libraries(quickstep_storage_BloomFilterIndexSubBlock quickstep_types_operations_comparisons_ComparisonID quickstep_utility_BloomFilter quickstep_utility_Macros) +# CMAKE_VALIDATE_IGNORE_BEGIN +if(QUICKSTEP_HAVE_BITWEAVING) + target_link_libraries(quickstep_storage_BitWeavingHIndexSubBlock + glog + quickstep_catalog_CatalogTypedefs + quickstep_compression_CompressionDictionary + quickstep_compression_CompressionDictionaryBuilder + quickstep_expressions_predicate_ComparisonPredicate + quickstep_expressions_predicate_PredicateCost + quickstep_storage_BitWeavingIndexSubBlock + quickstep_storage_CompressedTupleStorageSubBlock + quickstep_storage_StorageBlockInfo + quickstep_storage_StorageBlockLayout_proto + quickstep_storage_SubBlockTypeRegistry + quickstep_storage_SubBlockTypeRegistryMacros + quickstep_storage_TupleStorageSubBlock + quickstep_types_TypedValue + quickstep_types_operations_comparisons_ComparisonID + quickstep_utility_Macros) + target_link_libraries(quickstep_storage_BitWeavingIndexSubBlock + glog + quickstep_catalog_CatalogAttribute + quickstep_catalog_CatalogRelationSchema + quickstep_catalog_CatalogTypedefs + quickstep_compression_CompressionDictionary + quickstep_compression_CompressionDictionaryBuilder + quickstep_expressions_predicate_ComparisonPredicate + quickstep_expressions_predicate_Predicate + quickstep_expressions_scalar_Scalar + quickstep_expressions_scalar_ScalarAttribute + quickstep_storage_CompressedStoreUtil + quickstep_storage_CompressedTupleStorageSubBlock + quickstep_storage_IndexSubBlock + quickstep_storage_StorageBlockInfo + quickstep_storage_StorageBlockLayout_proto + quickstep_storage_StorageConstants + quickstep_storage_StorageErrors + quickstep_storage_TupleIdSequence + quickstep_storage_TupleStorageSubBlock + quickstep_storage_ValueAccessor + quickstep_storage_ValueAccessorUtil + quickstep_types_Type + quickstep_types_TypedValue + quickstep_types_operations_comparisons_ComparisonID + quickstep_utility_BitManipulation + quickstep_utility_Macros) + target_link_libraries(quickstep_storage_BitWeavingVIndexSubBlock + glog + quickstep_catalog_CatalogTypedefs + quickstep_compression_CompressionDictionary + quickstep_compression_CompressionDictionaryBuilder + quickstep_expressions_predicate_ComparisonPredicate + quickstep_expressions_predicate_PredicateCost + quickstep_storage_BitWeavingIndexSubBlock + quickstep_storage_CompressedTupleStorageSubBlock + quickstep_storage_StorageBlockInfo + quickstep_storage_StorageBlockLayout_proto + quickstep_storage_StorageConstants + quickstep_storage_SubBlockTypeRegistry + quickstep_storage_SubBlockTypeRegistryMacros + quickstep_storage_TupleStorageSubBlock + quickstep_storage_ValueAccessor + quickstep_storage_ValueAccessorUtil + quickstep_types_TypedValue + quickstep_types_operations_comparisons_ComparisonID + quickstep_utility_Macros) +endif() +# CMAKE_VALIDATE_IGNORE_END target_link_libraries(quickstep_storage_ColumnStoreUtil quickstep_catalog_CatalogAttribute quickstep_catalog_CatalogRelationSchema @@ -795,6 +876,14 @@ target_link_libraries(quickstep_storage_StorageBlock quickstep_types_operations_comparisons_ComparisonUtil quickstep_utility_Macros quickstep_utility_PtrVector) +# CMAKE_VALIDATE_IGNORE_BEGIN +if(QUICKSTEP_HAVE_BITWEAVING) + target_link_libraries(quickstep_storage_StorageBlock + quickstep_storage_BitWeavingHIndexSubBlock + quickstep_storage_BitWeavingIndexSubBlock + quickstep_storage_BitWeavingVIndexSubBlock) +endif() +# CMAKE_VALIDATE_IGNORE_END target_link_libraries(quickstep_storage_StorageBlockBase glog quickstep_storage_StorageBlockInfo @@ -956,6 +1045,14 @@ elseif (QUICKSTEP_HAVE_FILE_MANAGER_WINDOWS) target_link_libraries(quickstep_storage quickstep_storage_FileManagerWindows) endif() +# CMAKE_VALIDATE_IGNORE_BEGIN +if(QUICKSTEP_HAVE_BITWEAVING) + target_link_libraries(quickstep_storage + quickstep_storage_BitWeavingHIndexSubBlock + quickstep_storage_BitWeavingIndexSubBlock + quickstep_storage_BitWeavingVIndexSubBlock) +endif() +# CMAKE_VALIDATE_IGNORE_END # Tests: include(CheckTypeSize) @@ -1094,6 +1191,49 @@ target_link_libraries(BloomFilterIndexSubBlock_unittest ${LIBS}) add_test(BloomFilterIndexSubBlock_unittest BloomFilterIndexSubBlock_unittest) +if(QUICKSTEP_HAVE_BITWEAVING) + add_executable(BitWeavingIndexSubBlock_unittest + "${CMAKE_CURRENT_SOURCE_DIR}/bitweaving/tests/BitWeavingIndexSubBlock_unittest.cpp") + target_link_libraries(BitWeavingIndexSubBlock_unittest + glog + gtest + gtest_main + quickstep_catalog_CatalogAttribute + quickstep_catalog_CatalogRelation + quickstep_catalog_CatalogTypedefs + quickstep_expressions_predicate_ComparisonPredicate + quickstep_expressions_scalar_Scalar + quickstep_expressions_scalar_ScalarAttribute + quickstep_expressions_scalar_ScalarLiteral + quickstep_storage_BitWeavingHIndexSubBlock + quickstep_storage_BitWeavingIndexSubBlock + quickstep_storage_BitWeavingVIndexSubBlock + quickstep_storage_StorageBlock + quickstep_storage_StorageBlockInfo + quickstep_storage_StorageBlockLayout_proto + quickstep_storage_StorageErrors + quickstep_storage_StorageManager + quickstep_storage_TupleIdSequence + quickstep_storage_TupleStorageSubBlock + quickstep_types_CharType + quickstep_types_FloatType + quickstep_types_LongType + quickstep_types_Type + quickstep_types_TypeFactory + quickstep_types_TypeID + quickstep_types_TypedValue + quickstep_types_VarCharType + quickstep_types_containers_Tuple + quickstep_types_operations_comparisons_Comparison + quickstep_types_operations_comparisons_ComparisonFactory + quickstep_types_operations_comparisons_ComparisonID + quickstep_types_operations_comparisons_ComparisonUtil + quickstep_utility_Macros + quickstep_utility_ScopedBuffer + ${LIBS}) + add_test(BitWeavingIndexSubBlock_unittest BitWeavingIndexSubBlock_unittest) +endif() + add_executable(CompressedColumnStoreTupleStorageSubBlock_unittest "${CMAKE_CURRENT_SOURCE_DIR}/tests/CompressedColumnStoreTupleStorageSubBlock_unittest.cpp") target_link_libraries(CompressedColumnStoreTupleStorageSubBlock_unittest gtest http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/IndexSubBlockDescriptionFactory.hpp ---------------------------------------------------------------------- diff --git a/storage/IndexSubBlockDescriptionFactory.hpp b/storage/IndexSubBlockDescriptionFactory.hpp index a9992ef..5e7f5a1 100644 --- a/storage/IndexSubBlockDescriptionFactory.hpp +++ b/storage/IndexSubBlockDescriptionFactory.hpp @@ -28,7 +28,7 @@ namespace quickstep { */ /** - * @brief A class that describes factories of IndexSubBlockDescriptionFactory + * @brief A class that describes factories of IndexSubBlockDescriptionFactory. */ class IndexSubBlockDescriptionFactory { public: @@ -51,6 +51,13 @@ class IndexSubBlockDescriptionFactory { // Different types of indexes can specify their own validation checks here. // Currently these are trivial for CSBTree and SMAIndex. switch (index_description.sub_block_type()) { + case IndexSubBlockDescription_IndexSubBlockType_BITWEAVING_H: // Fall through. + case IndexSubBlockDescription_IndexSubBlockType_BITWEAVING_V: + // Only singular keys are allowed. + if (index_description.indexed_attribute_ids_size() == 1) { + return true; + } + return false; case IndexSubBlockDescription_IndexSubBlockType_CSB_TREE: return true; case IndexSubBlockDescription_IndexSubBlockType_SMA: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/StorageBlock.cpp ---------------------------------------------------------------------- diff --git a/storage/StorageBlock.cpp b/storage/StorageBlock.cpp index fba4d60..476a130 100644 --- a/storage/StorageBlock.cpp +++ b/storage/StorageBlock.cpp @@ -63,6 +63,12 @@ #include "glog/logging.h" +#ifdef QUICKSTEP_HAVE_BITWEAVING +#include "storage/bitweaving/BitWeavingIndexSubBlock.hpp" +#include "storage/bitweaving/BitWeavingHIndexSubBlock.hpp" +#include "storage/bitweaving/BitWeavingVIndexSubBlock.hpp" +#endif + using std::make_pair; using std::pair; using std::size_t; @@ -172,8 +178,7 @@ StorageBlock::StorageBlock(const CatalogRelationSchema &relation, ++index_num) { indices_.push_back(CreateIndexSubBlock(*tuple_store_, block_header_.layout().index_description(index_num), - new_block, - sub_block_address, + new_block, sub_block_address, block_header_.index_size(index_num))); sub_block_address += block_header_.index_size(index_num); if (!indices_.back().supportsAdHocAdd()) { @@ -1044,6 +1049,25 @@ IndexSubBlock* StorageBlock::CreateIndexSubBlock( new_block, sub_block_memory, sub_block_memory_size); +#ifdef QUICKSTEP_HAVE_BITWEAVING + case IndexSubBlockDescription::BITWEAVING_V: + return new BitWeavingVIndexSubBlock(tuple_store, + description, + new_block, + sub_block_memory, + sub_block_memory_size); + case IndexSubBlockDescription::BITWEAVING_H: + return new BitWeavingHIndexSubBlock(tuple_store, + description, + new_block, + sub_block_memory, + sub_block_memory_size); +#else + case IndexSubBlockDescription::BITWEAVING_V: // Fall through. + case IndexSubBlockDescription::BITWEAVING_H: + LOG(FATAL) << "Attempted to create a block with a bitweaving index " + << "but Quickstep was not compiled with bitweaving."; +#endif default: if (new_block) { FATAL_ERROR("A StorageBlockLayout provided an unknown IndexBlockType."); http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/StorageBlockInfo.hpp ---------------------------------------------------------------------- diff --git a/storage/StorageBlockInfo.hpp b/storage/StorageBlockInfo.hpp index ecda854..c18a8a2 100644 --- a/storage/StorageBlockInfo.hpp +++ b/storage/StorageBlockInfo.hpp @@ -137,6 +137,8 @@ enum IndexSubBlockType { kCSBTree = 0, kSMA, kBloomFilter, + kBitWeavingV, + kBitWeavingH, kNumIndexSubBlockTypes // Not an actual IndexSubBlockType, exists for counting purposes. }; http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/StorageBlockLayout.proto ---------------------------------------------------------------------- diff --git a/storage/StorageBlockLayout.proto b/storage/StorageBlockLayout.proto index 571c700..9e13fbe 100644 --- a/storage/StorageBlockLayout.proto +++ b/storage/StorageBlockLayout.proto @@ -65,6 +65,8 @@ message IndexSubBlockDescription { CSB_TREE = 0; SMA = 1; BLOOM_FILTER = 2; + BITWEAVING_H = 3; + BITWEAVING_V = 4; } required IndexSubBlockType sub_block_type = 1; http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/StorageConfig.h.in ---------------------------------------------------------------------- diff --git a/storage/StorageConfig.h.in b/storage/StorageConfig.h.in index 2344bba..eee1be9 100644 --- a/storage/StorageConfig.h.in +++ b/storage/StorageConfig.h.in @@ -15,6 +15,7 @@ * limitations under the License. **/ +#cmakedefine QUICKSTEP_HAVE_BITWEAVING #cmakedefine QUICKSTEP_HAVE_FILE_MANAGER_POSIX #cmakedefine QUICKSTEP_HAVE_FILE_MANAGER_WINDOWS #cmakedefine QUICKSTEP_HAVE_FILE_MANAGER_HDFS http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/SubBlockTypeRegistry.cpp ---------------------------------------------------------------------- diff --git a/storage/SubBlockTypeRegistry.cpp b/storage/SubBlockTypeRegistry.cpp index 35ee502..f519c73 100644 --- a/storage/SubBlockTypeRegistry.cpp +++ b/storage/SubBlockTypeRegistry.cpp @@ -82,6 +82,28 @@ bool SubBlockTypeRegistry::LayoutDescriptionIsValid( return true; } +bool SubBlockTypeRegistry::IndexDescriptionIsValid( + const CatalogRelationSchema &relation, + const IndexSubBlockDescription &description) { + if (!description.IsInitialized()) { + return false; + } + + std::unordered_map::const_iterator + index_it = Instance()->index_description_is_valid_functions_.find( + static_cast(description.sub_block_type())); + // No validity function was registered. + if (index_it == Instance()->index_description_is_valid_functions_.end()) { + return false; + } + + if (!(*index_it->second)(relation, description)) { + return false; + } + return true; +} + std::size_t SubBlockTypeRegistry::EstimateBytesPerTupleForTupleStore( const CatalogRelationSchema &relation, const TupleStorageSubBlockDescription &description) { http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/storage/SubBlockTypeRegistry.hpp ---------------------------------------------------------------------- diff --git a/storage/SubBlockTypeRegistry.hpp b/storage/SubBlockTypeRegistry.hpp index a223e7c..e930fb4 100644 --- a/storage/SubBlockTypeRegistry.hpp +++ b/storage/SubBlockTypeRegistry.hpp @@ -142,6 +142,10 @@ class SubBlockTypeRegistry { const CatalogRelationSchema &relation, const StorageBlockLayoutDescription &description); + static bool IndexDescriptionIsValid( + const CatalogRelationSchema &relation, + const IndexSubBlockDescription &description); + static std::size_t EstimateBytesPerTupleForTupleStore( const CatalogRelationSchema &relation, const TupleStorageSubBlockDescription &description); http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/cbd169e5/validate_cmakelists.py ---------------------------------------------------------------------- diff --git a/validate_cmakelists.py b/validate_cmakelists.py index a064a1c..c7b5883 100755 --- a/validate_cmakelists.py +++ b/validate_cmakelists.py @@ -42,12 +42,16 @@ EXCLUDED_TOP_LEVEL_DIRS = ["build", "third_party"] IGNORED_DEPENDENCIES = frozenset( ["quickstep_threading_WinThreadsAPI", "quickstep_utility_textbasedtest_TextBasedTest", - "quickstep_utility_textbasedtest_TextBasedTestDriver"]) + "quickstep_utility_textbasedtest_TextBasedTestDriver", + "quickstep_storage_bitweaving_BitWeavingHIndexSubBlock", + "quickstep_storage_bitweaving_BitWeavingIndexSubBlock", + "quickstep_storage_bitweaving_BitWeavingVIndexSubBlock"]) # States when scanning a CMakeLists.txt file. CMAKE_SCANNING_NONE = 0 CMAKE_SCANNING_LIBRARY = 1 CMAKE_SCANNING_TARGET_LINK_LIBRARIES = 2 +CMAKE_SCANNING_IGNORE = 3 def convert_path_to_targetname(include_path): """Convert an included header file's path to a quickstep library target in @@ -315,10 +319,26 @@ def process_cmakelists_file(cmakelists_filename, qs_module_dirs): skipped_targets = set() generated_targets = set() scan_state = CMAKE_SCANNING_NONE + previous_state = CMAKE_SCANNING_NONE stitched_string = "" with open(cmakelists_filename, "r") as cmakelists_file: for line in cmakelists_file: - if scan_state == CMAKE_SCANNING_NONE: + if ("CMAKE_VALIDATE_IGNORE_BEGIN" in line and + scan_state != CMAKE_SCANNING_IGNORE): + previous_state = scan_state + scan_state = CMAKE_SCANNING_IGNORE + continue + + if scan_state == CMAKE_SCANNING_IGNORE: + if "CMAKE_VALIDATE_IGNORE_END" in line: + scan_state = previous_state + elif "CMAKE_VALIDATE_IGNORE_BEGIN" in line: + print "Nested IGNORE_BEGIN directives found in: "\ + + cmakelists_filename + ", exiting" + exit(-1) + else: + continue + elif scan_state == CMAKE_SCANNING_NONE: add_library_pos = line.find("add_library(") if add_library_pos != -1: scan_state = CMAKE_SCANNING_LIBRARY