From commits-return-8078-archive-asf-public=cust-asf.ponee.io@kudu.apache.org Thu Nov 7 04:00:01 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id F1D35180661 for ; Thu, 7 Nov 2019 05:00:00 +0100 (CET) Received: (qmail 17333 invoked by uid 500); 7 Nov 2019 04:00:00 -0000 Mailing-List: contact commits-help@kudu.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kudu.apache.org Delivered-To: mailing list commits@kudu.apache.org Received: (qmail 17324 invoked by uid 99); 7 Nov 2019 04:00:00 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Nov 2019 04:00:00 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id E6640805E6; Thu, 7 Nov 2019 03:59:59 +0000 (UTC) Date: Thu, 07 Nov 2019 03:59:59 +0000 To: "commits@kudu.apache.org" Subject: [kudu] branch branch-1.10.x updated: KUDU-2990: remove memkind/libnuma and dynamically link memkind at runtime MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <157309919987.29218.2155042947351058382@gitbox.apache.org> From: adar@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: kudu X-Git-Refname: refs/heads/branch-1.10.x X-Git-Reftype: branch X-Git-Oldrev: 1924b0965229359d135a167631afcafa4f783794 X-Git-Newrev: 831847fe3cba94437daaa44a50aee67cdd828466 X-Git-Rev: 831847fe3cba94437daaa44a50aee67cdd828466 X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. adar pushed a commit to branch branch-1.10.x in repository https://gitbox.apache.org/repos/asf/kudu.git The following commit(s) were added to refs/heads/branch-1.10.x by this push: new 831847f KUDU-2990: remove memkind/libnuma and dynamically link memkind at runtime 831847f is described below commit 831847fe3cba94437daaa44a50aee67cdd828466 Author: Adar Dembo AuthorDate: Fri Nov 1 22:37:20 2019 -0700 KUDU-2990: remove memkind/libnuma and dynamically link memkind at runtime This patch removes memkind and libnuma from the thirdparty tree and changes the NVM cache implementation to dynamically link memkind at runtime using dlopen() and friends. KUDU-2990 explains why we need to do this in great detail so I won't repeat that here. The alternatives I considered: 1. Patch memkind to dlopen() libnuma. This is probably the most user-friendly approach because libnuma is found in most systems (or at least in most package repositories), but a new enough memkind is not, and having Kudu distribute memkind eases adoption of the NVM cache. However, I was nervous about performing deep surgery in memkind as it's a codebase with which I'm not familiar, and I wanted to minimize risk because we'll be backporting this patch to several older branches. 2. Patch memkind to define weak libnuma symbols. If done correctly, behavior is unchanged when libnuma is present on the host system, but if it's not there, calls from memkind to libnuma will crash. Again, I didn't feel comfortable hacking into memkind, plus I've found weak symbols difficult to use in the past. 3. Remove the NVM cache from Kudu altogether. This is obviously the safest and simplest option, but it punishes Kudu users who actually use the NVM cache. 4. Gate the build of the NVM cache behind a CMake option. This ought to satisfy the ASF's definition of an "optional" feature, but does nothing for binary redistributors who wish to offer the NVM cache and who build Kudu as a statically linked "fat" binary. 5. Build as we do today, but forbid static linkage of libnuma. Binary redistributors will need to choose between including libnuma in their distributions, or forcing Kudu to look for libnuma at runtime. The former still violates ASF policy, and the latter means Kudu won't start on a system lacking libnuma, regardless of whether the NVM cache is actually in use. So what are the ramifications of the chosen approach? - Kudu no longer distributes memkind or libnuma. To use the NVM cache, the host system must provide both, and memkind must be version 1.6.0 or newer. CentOS 6 and Ubuntu 18.04 repositories all carried memkind 1.1.0. CentOS 7 has memkind 1.7.0. Persistent memory hardware itself also has a pretty steep kernel version requirement, so it's unlikely to be found outside of a new distro in the first place. - Tests that exercise the NVM cache will be skipped if they can't find a conformant memkind (and libnuma). - When starting Kudu, if you don't set --block_cache_type=NVM, you shoudn't notice any change. - If you do, Kudu will crash at startup if it can't find a conformant memkind. This affects upgrades: if you were already an NVM cache user but you didn't have memkind installed, your Kudu will crash post-upgrade. Note: this doesn't preclude implementing alternative #1 (the one I think is ideal) in the future; we'll just have to revert the bulk of this patch when we do so. To test, I ran cfile-test and cache-test as follows: - Without memkind installed: DRAM tests passed, NVM tests were skipped - With an old memkind installed: DRAM tests passed, NVM tests were skipped - With LD_LIBRARY_PATH=/path/to/memkind-1.9.0: All tests passed I also manually ran a Kudu master with --block_cache_type=NVM and without memkind to verify the crashing behavior. I had to resolve conflicts in src/kudu/cfile/cfile-test.cc. Change-Id: I4f474196aa98b5fa6e5966b9a3aea9a7e466805c Reviewed-on: http://gerrit.cloudera.org:8080/14620 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin (cherry picked from commit ba908efa15774bb24b5bfa4ea88915161d1100d1) Reviewed-on: http://gerrit.cloudera.org:8080/14648 Reviewed-by: Grant Henke --- CMakeLists.txt | 24 ----- cmake_modules/FindMemkind.cmake | 38 ------- cmake_modules/FindNuma.cmake | 38 ------- src/kudu/cfile/CMakeLists.txt | 6 -- src/kudu/cfile/block_cache.cc | 8 +- src/kudu/cfile/cfile-test.cc | 59 ++++++++--- src/kudu/util/CMakeLists.txt | 31 +----- src/kudu/util/cache-test.cc | 57 +++++----- src/kudu/util/nvm_cache.cc | 116 +++++++++++++++++++-- src/kudu/util/nvm_cache.h | 13 +++ thirdparty/build-definitions.sh | 68 ------------ thirdparty/build-thirdparty.sh | 18 ---- thirdparty/download-thirdparty.sh | 15 --- .../memkind-fix-build-with-old-autoconf.patch | 32 ------ .../patches/memkind-fix-build-with-old-glibc.patch | 24 ----- ...kind-fix-jemalloc-build-with-old-autoconf.patch | 36 ------- thirdparty/vars.sh | 9 -- 17 files changed, 195 insertions(+), 397 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index d53f81e..3aa9107 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1139,30 +1139,6 @@ if (NOT "${KUDU_USE_ASAN}" AND set(KUDU_TCMALLOC_AVAILABLE 1) endif() -# Required memkind libraries -if (NOT APPLE) - find_package(Memkind REQUIRED) - include_directories(${MEMKIND_INCLUDE_DIR}) - ADD_THIRDPARTY_LIB(memkind - STATIC_LIB "${MEMKIND_STATIC_LIB}" - SHARED_LIB "${MEMKIND_SHARED_LIB}" - DEPS numa) -endif() - -if (EXISTS ${MEMKIND_SHARED_LIB}) - set(HAVE_LIB_MEMKIND 1) - add_definitions(-DHAVE_LIB_MEMKIND=1) -endif() - -## libnuma -if (NOT APPLE) - find_package(Numa REQUIRED) - include_directories(${NUMA_INCLUDE_DIR}) - ADD_THIRDPARTY_LIB(numa - STATIC_LIB "${NUMA_STATIC_LIB}" - SHARED_LIB "${NUMA_SHARED_LIB}") -endif() - ## curl find_package(CURL REQUIRED) diff --git a/cmake_modules/FindMemkind.cmake b/cmake_modules/FindMemkind.cmake deleted file mode 100644 index 7e4cd02..0000000 --- a/cmake_modules/FindMemkind.cmake +++ /dev/null @@ -1,38 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -# - Find required memkind libraries -# This module defines -# MEMKIND_INCLUDE_DIR, directory containing headers -# MEMKIND_STATIC_LIB, path to *.a -# MEMKIND_SHARED_LIB, path to *.so shared library - -find_path(MEMKIND_INCLUDE_DIR memkind.h - NO_CMAKE_SYSTEM_PATH - NO_SYSTEM_ENVIRONMENT_PATH) -find_library(MEMKIND_SHARED_LIB memkind - NO_CMAKE_SYSTEM_PATH - NO_SYSTEM_ENVIRONMENT_PATH) -find_library(MEMKIND_STATIC_LIB libmemkind.a - NO_CMAKE_SYSTEM_PATH - NO_SYSTEM_ENVIRONMENT_PATH) - -include(FindPackageHandleStandardArgs) -find_package_handle_standard_args(MEMKIND REQUIRED_VARS - MEMKIND_SHARED_LIB MEMKIND_STATIC_LIB - MEMKIND_INCLUDE_DIR -) diff --git a/cmake_modules/FindNuma.cmake b/cmake_modules/FindNuma.cmake deleted file mode 100644 index 2167b28..0000000 --- a/cmake_modules/FindNuma.cmake +++ /dev/null @@ -1,38 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -# - Find required numa libraries -# This module defines -# NUMA_INCLUDE_DIR, directory containing headers -# NUMA_STATIC_LIB, path to *.a -# NUMA_SHARED_LIB, path to *.so shared library - -find_path(NUMA_INCLUDE_DIR numa.h - NO_CMAKE_SYSTEM_PATH -NO_SYSTEM_ENVIRONMENT_PATH) -find_library(NUMA_SHARED_LIB numa - NO_CMAKE_SYSTEM_PATH - NO_SYSTEM_ENVIRONMENT_PATH) -find_library(NUMA_STATIC_LIB libnuma.a - NO_CMAKE_SYSTEM_PATH - NO_SYSTEM_ENVIRONMENT_PATH) - -include(FindPackageHandleStandardArgs) -find_package_handle_standard_args(NUMA REQUIRED_VARS - NUMA_SHARED_LIB NUMA_STATIC_LIB - NUMA_INCLUDE_DIR -) diff --git a/src/kudu/cfile/CMakeLists.txt b/src/kudu/cfile/CMakeLists.txt index e2cd369..5b72020 100644 --- a/src/kudu/cfile/CMakeLists.txt +++ b/src/kudu/cfile/CMakeLists.txt @@ -55,12 +55,6 @@ set(CFILE_LIBS cfile_proto bitshuffle) -if(HAVE_LIB_MEMKIND) - set(CFILE_LIBS - ${CFILE_LIBS} - nvm_cache) -endif() - target_link_libraries(cfile ${CFILE_LIBS}) # Tests diff --git a/src/kudu/cfile/block_cache.cc b/src/kudu/cfile/block_cache.cc index 5ea8316..a4b65c4 100644 --- a/src/kudu/cfile/block_cache.cc +++ b/src/kudu/cfile/block_cache.cc @@ -54,7 +54,9 @@ DEFINE_string(block_cache_type, "DRAM", "Which type of block cache to use for caching data. " "Valid choices are 'DRAM' or 'NVM'. DRAM, the default, " "caches data in regular memory. 'NVM' caches data " - "in a memory-mapped file using the memkind library."); + "in a memory-mapped file using the memkind library. To use 'NVM', " + "libmemkind 1.6.0 or newer must be available on the system; " + "otherwise Kudu will crash."); using strings::Substitute; @@ -75,12 +77,8 @@ Cache* CreateCache(int64_t capacity) { return NewCache( capacity, "block_cache"); case Cache::MemoryType::NVM: -#if defined(HAVE_LIB_MEMKIND) return NewCache( capacity, "block_cache"); -#else - LOG(FATAL) << "cache of NVM memory type is not supported"; -#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ... default: LOG(FATAL) << "unsupported LRU cache memory type: " << mem_type; return nullptr; diff --git a/src/kudu/cfile/cfile-test.cc b/src/kudu/cfile/cfile-test.cc index e5c4d51..3795126 100644 --- a/src/kudu/cfile/cfile-test.cc +++ b/src/kudu/cfile/cfile-test.cc @@ -70,6 +70,7 @@ #include "kudu/util/mem_tracker.h" #include "kudu/util/memory/arena.h" #include "kudu/util/metrics.h" +#include "kudu/util/nvm_cache.h" #include "kudu/util/slice.h" #include "kudu/util/status.h" #include "kudu/util/stopwatch.h" @@ -79,11 +80,8 @@ DECLARE_bool(cfile_write_checksums); DECLARE_bool(cfile_verify_checksums); DECLARE_string(block_cache_type); - -#if defined(__linux__) DECLARE_string(nvm_cache_path); DECLARE_bool(nvm_cache_simulate_allocation_failure); -#endif METRIC_DECLARE_counter(block_cache_hits_caching); @@ -402,7 +400,6 @@ class TestCFileBothCacheMemoryTypes : public ::testing::WithParamInterface { public: void SetUp() OVERRIDE { -#if defined(HAVE_LIB_MEMKIND) // The NVM cache can run using any directory as its path -- it doesn't have // a lot of practical use outside of an actual NVM device, but for testing // purposes, we'll point it at our test dir, unless otherwise specified. @@ -410,16 +407,13 @@ class TestCFileBothCacheMemoryTypes : FLAGS_nvm_cache_path = GetTestPath("nvm-cache"); ASSERT_OK(Env::Default()->CreateDir(FLAGS_nvm_cache_path)); } -#endif switch (GetParam()) { case Cache::MemoryType::DRAM: FLAGS_block_cache_type = "DRAM"; break; -#if defined(HAVE_LIB_MEMKIND) case Cache::MemoryType::NVM: FLAGS_block_cache_type = "NVM"; break; -#endif default: LOG(FATAL) << "Unknown block cache type: '" << static_cast(GetParam()); @@ -432,14 +426,9 @@ class TestCFileBothCacheMemoryTypes : } }; -#if defined(__linux__) INSTANTIATE_TEST_CASE_P(CacheMemoryTypes, TestCFileBothCacheMemoryTypes, ::testing::Values(Cache::MemoryType::DRAM, Cache::MemoryType::NVM)); -#else -INSTANTIATE_TEST_CASE_P(CacheMemoryTypes, TestCFileBothCacheMemoryTypes, - ::testing::Values(Cache::MemoryType::DRAM)); -#endif template void CopyOne(CFileIterator *it, @@ -459,6 +448,7 @@ void CopyOne(CFileIterator *it, // They take way too long with debugging enabled. TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileInts) { + RETURN_IF_NO_NVM_CACHE(GetParam()); BlockId block_id; LOG_TIMING(INFO, "writing 100m ints") { LOG(INFO) << "Starting writefile"; @@ -477,6 +467,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileInts) { } TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileNullableInts) { + RETURN_IF_NO_NVM_CACHE(GetParam()); BlockId block_id; LOG_TIMING(INFO, "writing 100m nullable ints") { LOG(INFO) << "Starting writefile"; @@ -495,18 +486,22 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileNullableInts) { } TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileStringsPrefixEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestWrite100MFileStrings(PREFIX_ENCODING); } TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MUniqueStringsDictEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestWrite100MFileStrings(DICT_ENCODING); } TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MLowCardinalityStringsDictEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestWriteDictEncodingLowCardinalityStrings(100 * 1e6); } TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileStringsPlainEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestWrite100MFileStrings(PLAIN_ENCODING); } @@ -514,6 +509,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileStringsPlainEncoding) { // Write and Read 1 million unique strings with dictionary encoding TEST_P(TestCFileBothCacheMemoryTypes, TestWrite1MUniqueFileStringsDictEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); BlockId block_id; LOG_TIMING(INFO, "writing 1M unique strings") { LOG(INFO) << "Starting writefile"; @@ -533,41 +529,49 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite1MUniqueFileStringsDictEncoding) // Write and Read 1 million strings, which contains duplicates with dictionary encoding TEST_P(TestCFileBothCacheMemoryTypes, TestWrite1MLowCardinalityStringsDictEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestWriteDictEncodingLowCardinalityStrings(1000000); } TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteUInt32) { + RETURN_IF_NO_NVM_CACHE(GetParam()); for (auto enc : { PLAIN_ENCODING, RLE }) { TestReadWriteFixedSizeTypes>(enc); } } TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteInt32) { + RETURN_IF_NO_NVM_CACHE(GetParam()); for (auto enc : { PLAIN_ENCODING, RLE }) { TestReadWriteFixedSizeTypes>(enc); } } TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteUInt64) { + RETURN_IF_NO_NVM_CACHE(GetParam()); for (auto enc : { PLAIN_ENCODING, RLE, BIT_SHUFFLE }) { TestReadWriteFixedSizeTypes>(enc); } } TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteInt64) { + RETURN_IF_NO_NVM_CACHE(GetParam()); for (auto enc : { PLAIN_ENCODING, RLE, BIT_SHUFFLE }) { TestReadWriteFixedSizeTypes>(enc); } } TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteInt128) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestReadWriteFixedSizeTypes>(PLAIN_ENCODING); } TEST_P(TestCFileBothCacheMemoryTypes, TestFixedSizeReadWritePlainEncodingFloat) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestReadWriteFixedSizeTypes>(PLAIN_ENCODING); } TEST_P(TestCFileBothCacheMemoryTypes, TestFixedSizeReadWritePlainEncodingDouble) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestReadWriteFixedSizeTypes>(PLAIN_ENCODING); } @@ -723,11 +727,13 @@ void TestCFile::TestReadWriteStrings(EncodingType encoding, TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteStringsPrefixEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestReadWriteStrings(PREFIX_ENCODING); } // Read/Write test for dictionary encoded blocks TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteStringsDictEncoding) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestReadWriteStrings(DICT_ENCODING); } @@ -738,6 +744,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteStringsDictEncoding) { // and runs extremely slowly with TSAN enabled. #ifndef THREAD_SANITIZER TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteLargeStrings) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + // Pad the values out to a length of ~65KB. // We use this method instead of just a longer sprintf format since // this is much more CPU-efficient (speeds up the test). @@ -756,6 +764,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteLargeStrings) { // Test that metadata entries stored in the cfile are persisted. TEST_P(TestCFileBothCacheMemoryTypes, TestMetadata) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + BlockId block_id; // Write the file. @@ -797,6 +807,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestMetadata) { } TEST_P(TestCFileBothCacheMemoryTypes, TestDefaultColumnIter) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + const int kNumItems = 64; uint8_t null_bitmap[BitmapSize(kNumItems)]; uint32_t data[kNumItems]; @@ -846,6 +858,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestDefaultColumnIter) { } TEST_P(TestCFileBothCacheMemoryTypes, TestAppendRaw) { + RETURN_IF_NO_NVM_CACHE(GetParam()); TestReadWriteRawBlocks(NO_COMPRESSION, 1000); TestReadWriteRawBlocks(SNAPPY, 1000); TestReadWriteRawBlocks(LZ4, 1000); @@ -853,6 +866,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestAppendRaw) { } TEST_P(TestCFileBothCacheMemoryTypes, TestChecksumFlags) { + RETURN_IF_NO_NVM_CACHE(GetParam()); for (bool write_checksums : {false, true}) { for (bool verify_checksums : {false, true}) { FLAGS_cfile_write_checksums = write_checksums; @@ -864,6 +878,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestChecksumFlags) { } TEST_P(TestCFileBothCacheMemoryTypes, TestDataCorruption) { + RETURN_IF_NO_NVM_CACHE(GetParam()); FLAGS_cfile_write_checksums = true; FLAGS_cfile_verify_checksums = true; @@ -901,6 +916,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestDataCorruption) { } TEST_P(TestCFileBothCacheMemoryTypes, TestNullInts) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + UInt32DataGenerator generator; TestNullTypes(&generator, PLAIN_ENCODING, NO_COMPRESSION); TestNullTypes(&generator, PLAIN_ENCODING, LZ4); @@ -911,18 +928,24 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestNullInts) { } TEST_P(TestCFileBothCacheMemoryTypes, TestNullFloats) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + FPDataGenerator generator; TestNullTypes(&generator, PLAIN_ENCODING, NO_COMPRESSION); TestNullTypes(&generator, BIT_SHUFFLE, NO_COMPRESSION); } TEST_P(TestCFileBothCacheMemoryTypes, TestNullPrefixStrings) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + StringDataGenerator generator("hello %zu"); TestNullTypes(&generator, PLAIN_ENCODING, NO_COMPRESSION); TestNullTypes(&generator, PLAIN_ENCODING, LZ4); } TEST_P(TestCFileBothCacheMemoryTypes, TestNullPlainStrings) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + StringDataGenerator generator("hello %zu"); TestNullTypes(&generator, PREFIX_ENCODING, NO_COMPRESSION); TestNullTypes(&generator, PREFIX_ENCODING, LZ4); @@ -930,12 +953,16 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestNullPlainStrings) { // Test for dictionary encoding TEST_P(TestCFileBothCacheMemoryTypes, TestNullDictStrings) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + StringDataGenerator generator("hello %zu"); TestNullTypes(&generator, DICT_ENCODING, NO_COMPRESSION); TestNullTypes(&generator, DICT_ENCODING, LZ4); } TEST_P(TestCFileBothCacheMemoryTypes, TestReleaseBlock) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + unique_ptr sink; ASSERT_OK(fs_manager_->CreateNewBlock({}, &sink)); ASSERT_EQ(WritableBlock::CLEAN, sink->state()); @@ -949,6 +976,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestReleaseBlock) { } TEST_P(TestCFileBothCacheMemoryTypes, TestLazyInit) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + // Create a small test file. BlockId block_id; { @@ -1000,6 +1029,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestLazyInit) { // different reader instances operating on the same block should use the same // block cache keys. TEST_P(TestCFileBothCacheMemoryTypes, TestCacheKeysAreStable) { + RETURN_IF_NO_NVM_CACHE(GetParam()); + // Set up block cache instrumentation. MetricRegistry registry; scoped_refptr entity(METRIC_ENTITY_server.Instantiate(®istry, "test_entity")); @@ -1039,14 +1070,14 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestCacheKeysAreStable) { } } -#if defined(HAVE_LIB_MEMKIND) // Inject failures in nvm allocation and ensure that we can still read a file. TEST_P(TestCFileBothCacheMemoryTypes, TestNvmAllocationFailure) { if (GetParam() != Cache::MemoryType::NVM) return; + RETURN_IF_NO_NVM_CACHE(GetParam()); + FLAGS_nvm_cache_simulate_allocation_failure = true; TestReadWriteFixedSizeTypes >(PLAIN_ENCODING); } -#endif class TestCFileDifferentCodecs : public TestCFile, public testing::WithParamInterface { diff --git a/src/kudu/util/CMakeLists.txt b/src/kudu/util/CMakeLists.txt index a9a56ce..05ad8c2 100644 --- a/src/kudu/util/CMakeLists.txt +++ b/src/kudu/util/CMakeLists.txt @@ -195,6 +195,7 @@ set(UTIL_SRCS net/net_util.cc net/sockaddr.cc net/socket.cc + nvm_cache.cc oid_generator.cc once.cc os-util.cc @@ -276,31 +277,6 @@ ADD_EXPORTABLE_LIBRARY(kudu_util EXPORTED_DEPS ${EXPORTED_UTIL_LIBS}) ####################################### -# nvm_cache -####################################### - -if(HAVE_LIB_MEMKIND) - set(NVM_CACHE_SRCS - nvm_cache.cc) - - set(NVM_CACHE_LIBS - gflags - glog - gutil - memkind) - - if(NOT APPLE) - set(NVM_CACHE_LIBS - ${NVM_CACHE_LIBS} - dl - rt) - endif() - - add_library(nvm_cache ${NVM_CACHE_SRCS}) - target_link_libraries(nvm_cache ${NVM_CACHE_LIBS}) -endif() - -####################################### # kudu_util_compression ####################################### set(UTIL_COMPRESSION_SRCS @@ -353,11 +329,6 @@ target_link_libraries(kudu_test_util gmock kudu_util) -if(HAVE_LIB_MEMKIND) - target_link_libraries(kudu_test_util - nvm_cache) -endif() - ####################################### # kudu_curl_util ####################################### diff --git a/src/kudu/util/cache-test.cc b/src/kudu/util/cache-test.cc index 66c78e9..6ba34b2 100644 --- a/src/kudu/util/cache-test.cc +++ b/src/kudu/util/cache-test.cc @@ -25,14 +25,13 @@ #include "kudu/util/faststring.h" #include "kudu/util/mem_tracker.h" #include "kudu/util/metrics.h" +#include "kudu/util/nvm_cache.h" #include "kudu/util/slice.h" #include "kudu/util/test_macros.h" #include "kudu/util/test_util.h" DECLARE_bool(cache_force_single_shard); -#if defined(HAVE_LIB_MEMKIND) DECLARE_string(nvm_cache_path); -#endif // #if defined(HAVE_LIB_MEMKIND) DECLARE_double(cache_memtracker_approximation_ratio); @@ -111,12 +110,10 @@ class CacheBaseTest : public KuduTest, FLAGS_cache_force_single_shard = (sharding_policy == ShardingPolicy::SingleShard); -#if defined(HAVE_LIB_MEMKIND) if (google::GetCommandLineFlagInfoOrDie("nvm_cache_path").is_default) { FLAGS_nvm_cache_path = GetTestPath("nvm-cache"); ASSERT_OK(Env::Default()->CreateDir(FLAGS_nvm_cache_path)); } -#endif // #if defined(HAVE_LIB_MEMKIND) switch (eviction_policy) { case Cache::EvictionPolicy::FIFO: @@ -136,13 +133,11 @@ class CacheBaseTest : public KuduTest, "cache_test")); break; case Cache::MemoryType::NVM: -#if defined(HAVE_LIB_MEMKIND) - cache_.reset(NewCache(cache_size(), - "cache_test")); -#else - FAIL() << "cache of NVM memory type is not supported"; -#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ... + if (CanUseNVMCacheForTests()) { + cache_.reset(NewCache(cache_size(), + "cache_test")); + } break; default: FAIL() << mem_type << ": unrecognized cache memory type"; @@ -161,10 +156,14 @@ class CacheBaseTest : public KuduTest, ASSERT_TRUE(mem_tracker_.get()); } - scoped_refptr entity = METRIC_ENTITY_server.Instantiate( - &metric_registry_, "test"); - unique_ptr metrics(new BlockCacheMetrics(entity)); - cache_->SetMetrics(std::move(metrics)); + // cache_ will be null if we're trying to set up a test for the NVM cache + // and were unable to do so. + if (cache_) { + scoped_refptr entity = METRIC_ENTITY_server.Instantiate( + &metric_registry_, "test"); + unique_ptr metrics(new BlockCacheMetrics(entity)); + cache_->SetMetrics(std::move(metrics)); + } } const size_t cache_size_; @@ -193,7 +192,6 @@ class CacheTest : } }; -#if defined(HAVE_LIB_MEMKIND) INSTANTIATE_TEST_CASE_P( CacheTypes, CacheTest, ::testing::Values( @@ -215,17 +213,9 @@ INSTANTIATE_TEST_CASE_P( make_tuple(Cache::MemoryType::NVM, Cache::EvictionPolicy::LRU, ShardingPolicy::SingleShard))); -#else -INSTANTIATE_TEST_CASE_P( - CacheTypes, CacheTest, - ::testing::Combine(::testing::Values(Cache::MemoryType::DRAM), - ::testing::Values(Cache::EvictionPolicy::FIFO, - Cache::EvictionPolicy::LRU), - ::testing::Values(ShardingPolicy::MultiShard, - ShardingPolicy::SingleShard))); -#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ... TEST_P(CacheTest, TrackMemory) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); if (mem_tracker_) { Insert(100, 100, 1); ASSERT_EQ(1, mem_tracker_->consumption()); @@ -236,6 +226,7 @@ TEST_P(CacheTest, TrackMemory) { } TEST_P(CacheTest, HitAndMiss) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); ASSERT_EQ(-1, Lookup(100)); Insert(100, 101); @@ -259,6 +250,7 @@ TEST_P(CacheTest, HitAndMiss) { } TEST_P(CacheTest, Erase) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); Erase(200); ASSERT_EQ(0, evicted_keys_.size()); @@ -278,6 +270,7 @@ TEST_P(CacheTest, Erase) { } TEST_P(CacheTest, EntriesArePinned) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); Insert(100, 101); auto h1 = cache_->Lookup(EncodeInt(100), Cache::EXPECT_IN_CACHE); ASSERT_EQ(101, DecodeInt(cache_->Value(h1))); @@ -306,6 +299,7 @@ TEST_P(CacheTest, EntriesArePinned) { // size of items still in the cache, which must be approximately the // same as the total capacity. TEST_P(CacheTest, HeavyEntries) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); const int kLight = cache_size() / 1000; const int kHeavy = cache_size() / 100; int added = 0; @@ -330,6 +324,7 @@ TEST_P(CacheTest, HeavyEntries) { } TEST_P(CacheTest, InvalidateAllEntries) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); constexpr const int kEntriesNum = 1024; // This scenarios assumes no evictions are done at the cache capacity. ASSERT_LE(kEntriesNum, cache_size()); @@ -353,6 +348,7 @@ TEST_P(CacheTest, InvalidateAllEntries) { } TEST_P(CacheTest, InvalidateNoEntries) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); constexpr const int kEntriesNum = 10; // This scenarios assumes no evictions are done at the cache capacity. ASSERT_LE(kEntriesNum, cache_size()); @@ -374,6 +370,7 @@ TEST_P(CacheTest, InvalidateNoEntries) { } TEST_P(CacheTest, InvalidateNoEntriesNoAdvanceIterationFunctor) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); constexpr const int kEntriesNum = 256; // This scenarios assumes no evictions are done at the cache capacity. ASSERT_LE(kEntriesNum, cache_size()); @@ -401,6 +398,7 @@ TEST_P(CacheTest, InvalidateNoEntriesNoAdvanceIterationFunctor) { } TEST_P(CacheTest, InvalidateOddKeyEntries) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); constexpr const int kEntriesNum = 64; // This scenarios assumes no evictions are done at the cache capacity. ASSERT_LE(kEntriesNum, cache_size()); @@ -506,22 +504,15 @@ class LRUCacheTest : } }; -#if defined(HAVE_LIB_MEMKIND) INSTANTIATE_TEST_CASE_P( CacheTypes, LRUCacheTest, ::testing::Combine(::testing::Values(Cache::MemoryType::DRAM, Cache::MemoryType::NVM), ::testing::Values(ShardingPolicy::MultiShard, ShardingPolicy::SingleShard))); -#else -INSTANTIATE_TEST_CASE_P( - CacheTypes, LRUCacheTest, - ::testing::Combine(::testing::Values(Cache::MemoryType::DRAM), - ::testing::Values(ShardingPolicy::MultiShard, - ShardingPolicy::SingleShard))); -#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ... TEST_P(LRUCacheTest, EvictionPolicy) { + RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam())); static constexpr int kNumElems = 1000; const int size_per_elem = cache_size() / kNumElems; diff --git a/src/kudu/util/nvm_cache.cc b/src/kudu/util/nvm_cache.cc index 320048b..8c186fd 100644 --- a/src/kudu/util/nvm_cache.cc +++ b/src/kudu/util/nvm_cache.cc @@ -19,6 +19,8 @@ #include "kudu/util/nvm_cache.h" +#include + #include #include #include @@ -31,7 +33,6 @@ #include #include #include -#include #include "kudu/gutil/atomic_refcount.h" #include "kudu/gutil/atomicops.h" @@ -43,13 +44,20 @@ #include "kudu/gutil/port.h" #include "kudu/gutil/ref_counted.h" #include "kudu/gutil/stl_util.h" +#include "kudu/gutil/strings/substitute.h" #include "kudu/gutil/sysinfo.h" #include "kudu/util/cache.h" #include "kudu/util/cache_metrics.h" #include "kudu/util/flag_tags.h" #include "kudu/util/locks.h" #include "kudu/util/metrics.h" +#include "kudu/util/scoped_cleanup.h" #include "kudu/util/slice.h" +#include "kudu/util/status.h" + +#ifndef MEMKIND_PMEM_MIN_SIZE +#define MEMKIND_PMEM_MIN_SIZE (1024 * 1024 * 16) // Taken from memkind 1.9.0. +#endif struct memkind; @@ -75,11 +83,93 @@ TAG_FLAG(nvm_cache_simulate_allocation_failure, unsafe); using std::string; using std::unique_ptr; using std::vector; +using strings::Substitute; namespace kudu { namespace { +// Taken together, these typedefs and this macro make it easy to call a +// memkind function: +// +// CALL_MEMKIND(memkind_malloc, vmp_, size); +typedef int (*memkind_create_pmem)(const char*, size_t, memkind**); +typedef int (*memkind_destroy_kind)(memkind*); +typedef void* (*memkind_malloc)(memkind*, size_t); +typedef size_t (*memkind_malloc_usable_size)(memkind*, void*); +typedef void (*memkind_free)(memkind*, void*); +#define CALL_MEMKIND(func_name, ...) ((func_name)g_##func_name)(__VA_ARGS__) + +// Function pointers into memkind; set by InitMemkindOps(). +void* g_memkind_create_pmem; +void* g_memkind_destroy_kind; +void* g_memkind_malloc; +void* g_memkind_malloc_usable_size; +void* g_memkind_free; + +// After InitMemkindOps() is called, true if memkind is available and safe +// to use, false otherwise. +bool g_memkind_available; + +std::once_flag g_memkind_ops_flag; + +// Try to dlsym() a particular symbol from 'handle', storing the result in 'ptr' +// if successful. +Status TryDlsym(void* handle, const char* sym, void** ptr) { + dlerror(); // Need to clear any existing error first. + void* ret = dlsym(handle, sym); + char* error = dlerror(); + if (error) { + return Status::NotSupported(Substitute("could not dlsym $0", sym), error); + } + *ptr = ret; + return Status::OK(); +} + +// Try to dlopen() memkind and set up all the function pointers we need from it. +// +// Note: in terms of protecting ourselves against changes in memkind, we'll +// notice (and fail) if a symbol is missing, but not if it's signature has +// changed or if there's some subtle behavioral change. A scan of the memkind +// repo suggests that backwards compatibility is enforced: symbols are only +// added and behavioral changes are effected via the introduction of new symbols. +void InitMemkindOps() { + g_memkind_available = false; + + // Use RTLD_NOW so that if any of memkind's dependencies aren't satisfied + // (e.g. libnuma is too old and is missing symbols), we'll know up front + // instead of during cache operations. + void* memkind_lib = dlopen("libmemkind.so.0", RTLD_NOW); + if (!memkind_lib) { + LOG(WARNING) << "could not dlopen: " << dlerror(); + return; + } + auto cleanup = MakeScopedCleanup([&]() { + dlclose(memkind_lib); + }); + +#define DLSYM_OR_RETURN(func_name, handle) do { \ + const Status _s = TryDlsym(memkind_lib, func_name, handle); \ + if (!_s.ok()) { \ + LOG(WARNING) << _s.ToString(); \ + return; \ + } \ + } while (0) + + DLSYM_OR_RETURN("memkind_create_pmem", &g_memkind_create_pmem); + DLSYM_OR_RETURN("memkind_destroy_kind", &g_memkind_destroy_kind); + DLSYM_OR_RETURN("memkind_malloc", &g_memkind_malloc); + DLSYM_OR_RETURN("memkind_malloc_usable_size", &g_memkind_malloc_usable_size); + DLSYM_OR_RETURN("memkind_free", &g_memkind_free); +#undef DLSYM_OR_RETURN + + g_memkind_available = true; + + // Need to keep the memkind library handle open so our function pointers + // remain loaded in memory. + cleanup.cancel(); +} + typedef simple_spinlock MutexType; // LRU cache implementation @@ -278,7 +368,7 @@ void* NvmLRUCache::MemkindMalloc(size_t size) { if (PREDICT_FALSE(FLAGS_nvm_cache_simulate_allocation_failure)) { return nullptr; } - return memkind_malloc(vmp_, size); + return CALL_MEMKIND(memkind_malloc, vmp_, size); } bool NvmLRUCache::Unref(LRUHandle* e) { @@ -295,7 +385,7 @@ void NvmLRUCache::FreeEntry(LRUHandle* e) { metrics_->cache_usage->DecrementBy(e->charge); metrics_->evictions->Increment(); } - memkind_free(vmp_, e); + CALL_MEMKIND(memkind_free, vmp_, e); } // Allocate nvm memory. Try until successful or FLAGS_nvm_cache_allocation_retry_count @@ -550,7 +640,7 @@ class ShardedLRUCache : public Cache { // Per the note at the top of this file, our cache is entirely volatile. // Hence, when the cache is destructed, we delete the underlying // memkind pool. - memkind_destroy_kind(vmp_); + CALL_MEMKIND(memkind_destroy_kind, vmp_); } virtual UniqueHandle Insert(UniquePendingHandle handle, @@ -606,7 +696,7 @@ class ShardedLRUCache : public Cache { handle->val_length = val_len; handle->key_length = key_len; handle->charge = (charge == kAutomaticCharge) ? - memkind_malloc_usable_size(vmp_, buf) : charge; + CALL_MEMKIND(memkind_malloc_usable_size, vmp_, buf) : charge; handle->hash = HashSlice(key); memcpy(handle->kv_data, key.data(), key.size()); return ph; @@ -617,7 +707,7 @@ class ShardedLRUCache : public Cache { } virtual void Free(PendingHandle* ph) OVERRIDE { - memkind_free(vmp_, ph); + CALL_MEMKIND(memkind_free, vmp_, ph); } size_t Invalidate(const InvalidationControl& ctl) override { @@ -634,6 +724,13 @@ class ShardedLRUCache : public Cache { template<> Cache* NewCache(size_t capacity, const std::string& id) { + std::call_once(g_memkind_ops_flag, InitMemkindOps); + + // TODO(adar): we should plumb the failure up the call stack, but at the time + // of writing the NVM cache is only usable by the block cache, and its use of + // the singleton pattern prevents the surfacing of errors. + CHECK(g_memkind_available) << "Memkind not available!"; + // memkind_create_pmem() will fail if the capacity is too small, but with // an inscrutable error. So, we'll check ourselves. CHECK_GE(capacity, MEMKIND_PMEM_MIN_SIZE) @@ -641,7 +738,7 @@ Cache* NewCache diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh index ff29e67..02c4bb6 100644 --- a/thirdparty/build-definitions.sh +++ b/thirdparty/build-definitions.sh @@ -759,74 +759,6 @@ build_trace_viewer() { cp -a $TRACE_VIEWER_SOURCE/tracing.* $TP_DIR/../www/ } -build_numactl() { - local BUILD_TYPE=$1 - - NUMACTL_BDIR=$TP_BUILD_DIR/$NUMACTL_NAME$MODE_SUFFIX - mkdir -p $NUMACTL_BDIR - pushd $NUMACTL_BDIR - - # Setting -fsanitize=thread only in CFLAGS does not work for compiling numactl, - # We also need to set it in LDFLAGS. - case $BUILD_TYPE in - "normal") - ;; - "tsan") - SANITIZER_THREAD_OPTION="-fsanitize=thread -L$PREFIX/lib -Wl,-rpath=$PREFIX/lib" - ;; - *) - echo "Unknown build type: $BUILD_TYPE" - exit 1 - ;; - esac - - CFLAGS="$EXTRA_CFLAGS" \ - LDFLAGS="$EXTRA_LDFLAGS $SANITIZER_THREAD_OPTION" \ - CPPFLAGS="$EXTRA_CPPFLAGS" \ - LIBS="$EXTRA_LIBS" \ - $NUMACTL_SOURCE/configure --prefix=$PREFIX - - make -j$PARALLEL $EXTRA_MAKEFLAGS install - popd -} - -build_memkind() { - MEMKIND_BDIR=$TP_BUILD_DIR/$MEMKIND_NAME$MODE_SUFFIX - mkdir -p $MEMKIND_BDIR - pushd $MEMKIND_BDIR - - # It doesn't appear possible to isolate source and build directories, so just - # prepopulate the latter using the former. - rsync -av --delete $MEMKIND_SOURCE/ . - - # We separate the build steps from build.sh in memkind source code - # since we need to avoid building the tests code with thread sanitizer - export JE_PREFIX=jemk_ - - # jemalloc is built in memkind and need to be compiled firstly. Here we disable c++ - # integration to prevent new and delete operator implementations from exported in kudu client - $MEMKIND_BDIR/build_jemalloc.sh --disable-cxx --prefix=$PREFIX - - $MEMKIND_BDIR/autogen.sh - CFLAGS="$EXTRA_CFLAGS -O2" \ - CPPFLAGS="$EXTRA_CPPFLAGS -I$PREFIX/include" \ - LDFLAGS="$EXTRA_LDFLAGS -L$PREFIX/lib -Wl,-rpath=$PREFIX/lib" \ - LIBS="$EXTRA_LIBS" \ - $MEMKIND_BDIR/configure --prefix=$PREFIX - - # Building unit tests and memkind examples in memkind with clang will prevent some - # header files from being included which will lead to failures. So we do a minimal - # installation here. - make -j$PARALLEL $EXTRA_MAKEFLAGS static_lib - make -j$PARALLEL $EXTRA_MAKEFLAGS install-exec install-data - # install jemalloc header files - cd $MEMKIND_BDIR/jemalloc/obj && make -j$PARALLEL $EXTRA_MAKEFLAGS install_include - - unset JE_PREFIX - - popd -} - build_boost() { local BUILD_TYPE=$1 pushd $BOOST_SOURCE diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh index 169300d..347cd1d 100755 --- a/thirdparty/build-thirdparty.sh +++ b/thirdparty/build-thirdparty.sh @@ -90,8 +90,6 @@ else "libunwind") F_LIBUNWIND=1 ;; "llvm") F_LLVM=1 ;; "trace-viewer") F_TRACE_VIEWER=1 ;; - "numactl") F_NUMACTL=1 ;; - "memkind") F_MEMKIND=1 ;; "boost") F_BOOST=1 ;; "breakpad") F_BREAKPAD=1 ;; "sparsehash") F_SPARSEHASH=1 ;; @@ -307,14 +305,6 @@ if [ -n "$F_UNINSTRUMENTED" -o -n "$F_CURL" ]; then build_curl fi -if [ -n "$OS_LINUX" ] && [ -n "$F_UNINSTRUMENTED" -o -n "$F_NUMACTL" ]; then - build_numactl normal -fi - -if [ -n "$OS_LINUX" ] && [ -n "$F_UNINSTRUMENTED" -o -n "$F_MEMKIND" ]; then - build_memkind -fi - restore_env ### Build C++ dependencies without instrumentation @@ -466,14 +456,6 @@ if [ -n "$F_TSAN" -o -n "$F_CURL" ]; then build_curl fi -if [ -n "$OS_LINUX" ] && [ -n "$F_TSAN" -o -n "$F_NUMACTL" ]; then - build_numactl tsan -fi - -if [ -n "$OS_LINUX" ] && [ -n "$F_TSAN" -o -n "$F_MEMKIND" ]; then - build_memkind -fi - restore_env ### Build C++ dependencies with TSAN instrumentation diff --git a/thirdparty/download-thirdparty.sh b/thirdparty/download-thirdparty.sh index dd51678..65c3964 100755 --- a/thirdparty/download-thirdparty.sh +++ b/thirdparty/download-thirdparty.sh @@ -348,21 +348,6 @@ fetch_and_patch \ $TRACE_VIEWER_SOURCE \ $TRACE_VIEWER_PATCHLEVEL -NUMACTL_PATCHLEVEL=0 -fetch_and_patch \ - numactl-${NUMACTL_VERSION}.tar.gz \ - $NUMACTL_SOURCE \ - $NUMACTL_PATCHLEVEL - -MEMKIND_PATCHLEVEL=3 -fetch_and_patch \ - memkind-${MEMKIND_VERSION}.tar.gz \ - $MEMKIND_SOURCE \ - $MEMKIND_PATCHLEVEL \ - "patch -p1 < $TP_DIR/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch" \ - "patch -p1 < $TP_DIR/patches/memkind-fix-build-with-old-autoconf.patch" \ - "patch -p1 < $TP_DIR/patches/memkind-fix-build-with-old-glibc.patch" - BOOST_PATCHLEVEL=1 fetch_and_patch \ boost_${BOOST_VERSION}.tar.gz \ diff --git a/thirdparty/patches/memkind-fix-build-with-old-autoconf.patch b/thirdparty/patches/memkind-fix-build-with-old-autoconf.patch deleted file mode 100644 index 0f99b98..0000000 --- a/thirdparty/patches/memkind-fix-build-with-old-autoconf.patch +++ /dev/null @@ -1,32 +0,0 @@ -commit c8dbc9f -Author: Adar Dembo -Date: Sun May 26 14:20:51 2019 -0700 - - configure.ac: fixes for autoconf 2.63 - -diff --git a/configure.ac b/configure.ac -index 64c3200..ec5dbb1 100644 ---- a/configure.ac -+++ b/configure.ac -@@ -25,10 +25,10 @@ - # -*- Autoconf -*- - # Process this file with autoconf to produce a configure script. - --AC_PREREQ([2.64]) -+AC_PREREQ([2.63]) - AC_INIT([memkind],m4_esyscmd([tr -d '\n' < VERSION])) - --AC_CONFIG_MACRO_DIRS([m4]) -+AC_CONFIG_MACRO_DIR([m4]) - AC_CONFIG_HEADERS([config.h]) - AC_CONFIG_SRCDIR([memkind.spec.mk]) - -@@ -36,7 +36,7 @@ AM_INIT_AUTOMAKE([-Wall -Werror foreign 1.11 silent-rules subdir-objects paralle - AM_SILENT_RULES([yes]) - - # Checks for programs and libraries. --AM_PROG_AR -+m4_ifdef([AM_PROG_AR], [AM_PROG_AR]) - AC_PROG_CXX - AC_PROG_CC - AC_OPENMP diff --git a/thirdparty/patches/memkind-fix-build-with-old-glibc.patch b/thirdparty/patches/memkind-fix-build-with-old-glibc.patch deleted file mode 100644 index 06dd525..0000000 --- a/thirdparty/patches/memkind-fix-build-with-old-glibc.patch +++ /dev/null @@ -1,24 +0,0 @@ -commit 2113b6b -Author: Adar Dembo -Date: Sun May 26 14:39:46 2019 -0700 - - Makefile.am: fixes for building against older glibc - - In older versions of glibc (such as the version found on el6 machines), - the clock_gettime function is in librt rather than in libc directly. Our - bundled jemalloc depends on clock_gettime and may link against librt, so - let's make sure the memkind library links against librt too. - -diff --git a/Makefile.am b/Makefile.am -index f791ce7..de57d90 100644 ---- a/Makefile.am -+++ b/Makefile.am -@@ -44,7 +44,7 @@ libmemkind_la_SOURCES = src/hbwmalloc.c \ - - - libmemkind_la_LIBADD = jemalloc/obj/lib/libjemalloc_pic.a --libmemkind_la_LDFLAGS = -version-info 0:1:0 -ldl -+libmemkind_la_LDFLAGS = -version-info 0:1:0 -ldl -lrt - include_HEADERS = include/hbw_allocator.h \ - include/hbwmalloc.h \ - include/memkind.h \ diff --git a/thirdparty/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch b/thirdparty/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch deleted file mode 100644 index fc6129f..0000000 --- a/thirdparty/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch +++ /dev/null @@ -1,36 +0,0 @@ -commit 608c2eb -Author: Adar Dembo -Date: Sun May 26 14:00:50 2019 -0700 - - jemalloc: unroll dlsym checking logic in configure.ac - - The nested calls here cause autoconf 2.63 to generate a broken script[1]. - The upstream jemalloc response has been to require autoconf 2.68, but that - means we can't build jemalloc (and thus memkind) on el6.6. As a workaround, - we can simply unroll these nested calls. - - 1. https://github.com/jemalloc/jemalloc/issues/912 - -diff --git a/jemalloc/configure.ac b/jemalloc/configure.ac -index 5bd5442..cbc1e5d 100644 ---- a/jemalloc/configure.ac -+++ b/jemalloc/configure.ac -@@ -1454,10 +1454,14 @@ if test "x$abi" != "xpecoff" ; then - have_pthread="1" - dnl Check if we have dlsym support. - have_dlsym="1" -- AC_CHECK_HEADERS([dlfcn.h], -- AC_CHECK_FUNC([dlsym], [], -- [AC_CHECK_LIB([dl], [dlsym], [LIBS="$LIBS -ldl"], [have_dlsym="0"])]), -- [have_dlsym="0"]) -+ AC_CHECK_HEADERS([dlfcn.h], , [have_dlsym="0"]) -+ check_dlsym_in_libdl="0" -+ if test "x$have_dlsym" = "x1" ; then -+ AC_CHECK_FUNC([dlsym], [], [check_dlsym_in_libdl="1"]) -+ fi -+ if test "x$check_dlsym_in_libdl" = "x1" ; then -+ AC_CHECK_LIB([dl], [dlsym], [LIBS="$LIBS -ldl"], [have_dlsym="0"]) -+ fi - if test "x$have_dlsym" = "x1" ; then - AC_DEFINE([JEMALLOC_HAVE_DLSYM], [ ]) - fi diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh index 2e01115..2a501a6 100644 --- a/thirdparty/vars.sh +++ b/thirdparty/vars.sh @@ -166,15 +166,6 @@ TRACE_VIEWER_VERSION=21d76f8350fea2da2aa25cb6fd512703497d0c11 TRACE_VIEWER_NAME=kudu-trace-viewer-$TRACE_VIEWER_VERSION TRACE_VIEWER_SOURCE=$TP_SOURCE_DIR/$TRACE_VIEWER_NAME -# numactl 2.0.12 is required to build memkind library. -NUMACTL_VERSION=2.0.12 -NUMACTL_NAME=numactl-$NUMACTL_VERSION -NUMACTL_SOURCE=$TP_SOURCE_DIR/$NUMACTL_NAME - -MEMKIND_VERSION=1.9.0 -MEMKIND_NAME=memkind-$MEMKIND_VERSION -MEMKIND_SOURCE=$TP_SOURCE_DIR/$MEMKIND_NAME - BOOST_VERSION=1_61_0 BOOST_NAME=boost_$BOOST_VERSION BOOST_SOURCE=$TP_SOURCE_DIR/$BOOST_NAME