Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 2306F200BE9 for ; Mon, 21 Nov 2016 04:58:33 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 20E1D160B07; Mon, 21 Nov 2016 03:58:33 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 24591160B1B for ; Mon, 21 Nov 2016 04:58:31 +0100 (CET) Received: (qmail 6549 invoked by uid 500); 21 Nov 2016 03:58:31 -0000 Mailing-List: contact commits-help@quickstep.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@quickstep.incubator.apache.org Delivered-To: mailing list commits@quickstep.incubator.apache.org Received: (qmail 6533 invoked by uid 99); 21 Nov 2016 03:58:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Nov 2016 03:58:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D6167C23EF for ; Mon, 21 Nov 2016 03:58:30 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.219 X-Spam-Level: X-Spam-Status: No, score=-6.219 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id JG6f2NTWDpPF for ; Mon, 21 Nov 2016 03:58:29 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 3EE545FCE0 for ; Mon, 21 Nov 2016 03:58:28 +0000 (UTC) Received: (qmail 6458 invoked by uid 99); 21 Nov 2016 03:58:27 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Nov 2016 03:58:27 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 543A8F17BB; Mon, 21 Nov 2016 03:58:27 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: zuyuz@apache.org To: commits@quickstep.incubator.apache.org Date: Mon, 21 Nov 2016 03:58:33 -0000 Message-Id: <38326f8e155943a79e8a4dc9232d4cdb@git.apache.org> In-Reply-To: <5574fbd15fd84719a6945dde8d4135ef@git.apache.org> References: <5574fbd15fd84719a6945dde8d4135ef@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [7/8] incubator-quickstep git commit: Moved a CLI flag into a new file. archived-at: Mon, 21 Nov 2016 03:58:33 -0000 Moved a CLI flag into a new file. Project: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/commit/f7d15436 Tree: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/tree/f7d15436 Diff: http://git-wip-us.apache.org/repos/asf/incubator-quickstep/diff/f7d15436 Branch: refs/heads/get-block-domain Commit: f7d1543671a3d3b3e52d2fe0c8170b64796aa9d6 Parents: e7c664b Author: Zuyu Zhang Authored: Sat Nov 19 00:18:55 2016 -0800 Committer: Zuyu Zhang Committed: Sun Nov 20 19:56:10 2016 -0800 ---------------------------------------------------------------------- CMakeLists.txt | 2 ++ cli/CMakeLists.txt | 5 +++++ cli/Flags.cpp | 43 +++++++++++++++++++++++++++++++++++++++++++ cli/Flags.hpp | 41 +++++++++++++++++++++++++++++++++++++++++ cli/QuickstepCli.cpp | 23 +++++++++-------------- 5 files changed, 100 insertions(+), 14 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/f7d15436/CMakeLists.txt ---------------------------------------------------------------------- diff --git a/CMakeLists.txt b/CMakeLists.txt index 6191de0..cd53967 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -752,6 +752,7 @@ target_link_libraries(quickstep_cli_shell quickstep_cli_CommandExecutor quickstep_cli_DefaultsConfigurator quickstep_cli_DropRelation + quickstep_cli_Flags quickstep_cli_InputParserUtil quickstep_cli_LineReader quickstep_cli_PrintToScreen @@ -770,6 +771,7 @@ target_link_libraries(quickstep_cli_shell quickstep_queryoptimizer_QueryProcessor quickstep_storage_PreloaderThread quickstep_storage_StorageConstants + quickstep_storage_StorageManager quickstep_threading_ThreadIDBasedMap quickstep_utility_ExecutionDAGVisualizer quickstep_utility_Macros http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/f7d15436/cli/CMakeLists.txt ---------------------------------------------------------------------- diff --git a/cli/CMakeLists.txt b/cli/CMakeLists.txt index b86821a..9928f3f 100644 --- a/cli/CMakeLists.txt +++ b/cli/CMakeLists.txt @@ -54,6 +54,7 @@ target_link_libraries(quickstep_cli_DropRelation quickstep_storage_StorageBlockInfo quickstep_storage_StorageManager quickstep_utility_Macros) +add_library(quickstep_cli_Flags Flags.cpp Flags.hpp) if(USE_LINENOISE) add_library(quickstep_cli_LineReader @@ -110,6 +111,9 @@ if(QUICKSTEP_HAVE_LIBNUMA) target_link_libraries(quickstep_cli_DefaultsConfigurator ${LIBNUMA_LIBRARY}) endif() +target_link_libraries(quickstep_cli_Flags + quickstep_storage_StorageConstants + ${GFLAGS_LIB_NAME}) target_link_libraries(quickstep_cli_InputParserUtil glog quickstep_utility_Macros @@ -139,6 +143,7 @@ target_link_libraries(quickstep_cli quickstep_cli_CommandExecutor quickstep_cli_DefaultsConfigurator quickstep_cli_DropRelation + quickstep_cli_Flags quickstep_cli_InputParserUtil quickstep_cli_LineReader quickstep_cli_PrintToScreen) http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/f7d15436/cli/Flags.cpp ---------------------------------------------------------------------- diff --git a/cli/Flags.cpp b/cli/Flags.cpp new file mode 100644 index 0000000..384ef71 --- /dev/null +++ b/cli/Flags.cpp @@ -0,0 +1,43 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + **/ + +#include "cli/Flags.hpp" + +#include + +#include "storage/StorageConstants.hpp" + +#include "gflags/gflags.h" + +namespace quickstep { + +static bool ValidateStoragePath(const char *flagname, + const std::string &value) { + if (!value.empty() && value.back() != kPathSeparator) { + FLAGS_storage_path.push_back(kPathSeparator); + } + + return true; +} +DEFINE_string(storage_path, kDefaultStoragePath, + "Filesystem path to store the Quickstep database."); +static const volatile bool storage_path_dummy + = gflags::RegisterFlagValidator(&FLAGS_storage_path, &ValidateStoragePath); + +} // namespace quickstep http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/f7d15436/cli/Flags.hpp ---------------------------------------------------------------------- diff --git a/cli/Flags.hpp b/cli/Flags.hpp new file mode 100644 index 0000000..a623448 --- /dev/null +++ b/cli/Flags.hpp @@ -0,0 +1,41 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + **/ + +#ifndef QUICKSTEP_CLI_FLAGS_HPP_ +#define QUICKSTEP_CLI_FLAGS_HPP_ + +#include "gflags/gflags_declare.h" + +namespace quickstep { + +/** \addtogroup CLI + * @{ + */ + +/** + * @brief A collection of common flags shared by Quickstep CLIs in both the + * single-node and the distributed version. + **/ +DECLARE_string(storage_path); + +/** @} */ + +} // namespace quickstep + +#endif // QUICKSTEP_CLI_FLAGS_HPP_ http://git-wip-us.apache.org/repos/asf/incubator-quickstep/blob/f7d15436/cli/QuickstepCli.cpp ---------------------------------------------------------------------- diff --git a/cli/QuickstepCli.cpp b/cli/QuickstepCli.cpp index a4e55d7..33a9ff4 100644 --- a/cli/QuickstepCli.cpp +++ b/cli/QuickstepCli.cpp @@ -54,6 +54,7 @@ typedef quickstep::LineReaderDumb LineReaderImpl; #endif #include "cli/DefaultsConfigurator.hpp" +#include "cli/Flags.hpp" #include "cli/InputParserUtil.hpp" #include "cli/PrintToScreen.hpp" #include "parser/ParseStatement.hpp" @@ -76,6 +77,7 @@ typedef quickstep::LineReaderDumb LineReaderImpl; #include "storage/PreloaderThread.hpp" #include "storage/StorageConstants.hpp" +#include "storage/StorageManager.hpp" #include "threading/ThreadIDBasedMap.hpp" #include "utility/ExecutionDAGVisualizer.hpp" #include "utility/Macros.hpp" @@ -107,6 +109,7 @@ using quickstep::AdmitRequestMessage; using quickstep::CatalogRelation; using quickstep::DefaultsConfigurator; using quickstep::DropRelation; +using quickstep::FLAGS_storage_path; using quickstep::ForemanSingleNode; using quickstep::InputParserUtil; using quickstep::MessageBusImpl; @@ -142,8 +145,6 @@ DEFINE_bool(preload_buffer_pool, false, "If true, pre-load all known blocks into buffer pool before " "accepting queries (should also set --buffer_pool_slots to be " "large enough to accomodate the entire database)."); -DEFINE_string(storage_path, kDefaultStoragePath, - "Filesystem path to store the Quickstep database."); DEFINE_string(worker_affinities, "", "A comma-separated list of CPU IDs to pin worker threads to " "(leaving this empty will cause all worker threads to inherit " @@ -231,15 +232,9 @@ int main(int argc, char* argv[]) { bus.RegisterClientAsSender(main_thread_client_id, kPoisonMessage); bus.RegisterClientAsReceiver(main_thread_client_id, kWorkloadCompletionMessage); - // Setup the paths used by StorageManager. - string fixed_storage_path(quickstep::FLAGS_storage_path); - if (!fixed_storage_path.empty() - && (fixed_storage_path.back() != quickstep::kPathSeparator)) { - fixed_storage_path.push_back(quickstep::kPathSeparator); - } - quickstep::StorageManager storage_manager(fixed_storage_path); + quickstep::StorageManager storage_manager(FLAGS_storage_path); - string catalog_path(fixed_storage_path); + string catalog_path(FLAGS_storage_path); catalog_path.append(kCatalogFilename); if (quickstep::FLAGS_initialize_db) { // Initialize the database // TODO(jmp): Refactor the code in this file! @@ -249,14 +244,14 @@ int main(int argc, char* argv[]) { // TODO(jmp): At some point, likely in C++-17, we will just have the // filesystem path, and we can clean this up #ifdef QUICKSTEP_OS_WINDOWS - std::filesystem::create_directories(fixed_storage_path); - LOG(FATAL) << "Failed when attempting to create the directory: " << fixed_storage_path << "\n"; + std::filesystem::create_directories(FLAGS_storage_path); + LOG(FATAL) << "Failed when attempting to create the directory: " << FLAGS_storage_path << "\n"; LOG(FATAL) << "Check if the directory already exists. If so, delete it or move it before initializing \n"; #else { - string path_name = "mkdir " + fixed_storage_path; + string path_name = "mkdir " + FLAGS_storage_path; if (std::system(path_name.c_str())) { - LOG(FATAL) << "Failed when attempting to create the directory: " << fixed_storage_path << "\n"; + LOG(FATAL) << "Failed when attempting to create the directory: " << FLAGS_storage_path << "\n"; } } #endif