kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From granthe...@apache.org
Subject [2/2] kudu git commit: thirdparty: patch tcmalloc to improve AllocLarge performance
Date Thu, 22 Feb 2018 20:40:51 GMT
thirdparty: patch tcmalloc to improve AllocLarge performance

This pulls in an upstream tcmalloc pull request[1] which addresses O(n)
behavior in the large allocation path. Without this patch, large
allocations (>=1MB) do a linear scan of all large spans of free heap
to find the best fit, which is very expensive especially as the heap
grows more fragmented over time.

I tested this using YCSB workload C (100% random read) on an in-memory dataset
(10M rows on 6 nodes). As noted in KUDU-1465, scanners currently always
allocate 1MB buffers as a starting point even if the scan will only return a
small amount of data. These 1M allocations aggravate the AllocLarge path
and without this patch, AllocLarge was the top CPU consumer.

Prior to this fix, the workload managed about 20k reads/second. With the
fix, the workload averaged around 85k reads/second.

We should still fix KUDU-1465 to allocate smaller buffers, since small
allocations will always be faster (and less wasteful), but with this tcmalloc
fix, the performance issue is much less pronounced. The fix will also likely
help any other places that we might be making large allocations.

[1] https://github.com/gperftools/gperftools/pull/960

Change-Id: I4abffbd1cb02f99ebda1628d98e5c342ccb7f0f9
Reviewed-on: http://gerrit.cloudera.org:8080/9392
Tested-by: Kudu Jenkins
Reviewed-by: Dan Burkert <dan@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/1ae56048
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/1ae56048
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/1ae56048

Branch: refs/heads/master
Commit: 1ae56048b8ad62f1f7d11a4e6bdd290f9c753191
Parents: 68d797d
Author: Todd Lipcon <todd@apache.org>
Authored: Thu Feb 22 00:28:59 2018 -0800
Committer: Todd Lipcon <todd@apache.org>
Committed: Thu Feb 22 20:15:52 2018 +0000

----------------------------------------------------------------------
 thirdparty/download-thirdparty.sh               |   3 +-
 ...ed-O-log-n-searching-among-large-spans.patch | 494 +++++++++++++++++++
 ...namespace-base-with-namespace-tcmalloc.patch | 125 +++--
 3 files changed, 571 insertions(+), 51 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/1ae56048/thirdparty/download-thirdparty.sh
----------------------------------------------------------------------
diff --git a/thirdparty/download-thirdparty.sh b/thirdparty/download-thirdparty.sh
index ba76d12..5ef1618 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -124,12 +124,13 @@ if [ ! -d $GFLAGS_SOURCE ]; then
   fetch_and_expand gflags-${GFLAGS_VERSION}.tar.gz
 fi
 
-GPERFTOOLS_PATCHLEVEL=2
+GPERFTOOLS_PATCHLEVEL=4
 delete_if_wrong_patchlevel $GPERFTOOLS_SOURCE $GPERFTOOLS_PATCHLEVEL
 if [ ! -d $GPERFTOOLS_SOURCE ]; then
   fetch_and_expand gperftools-${GPERFTOOLS_VERSION}.tar.gz
 
   pushd $GPERFTOOLS_SOURCE
+  patch -p1 < $TP_DIR/patches/gperftools-Implemented-O-log-n-searching-among-large-spans.patch
   patch -p1 < $TP_DIR/patches/gperftools-Replace-namespace-base-with-namespace-tcmalloc.patch
   patch -p1 < $TP_DIR/patches/gperftools-sized-alloc-build-fix.patch
   touch patchlevel-$GPERFTOOLS_PATCHLEVEL

http://git-wip-us.apache.org/repos/asf/kudu/blob/1ae56048/thirdparty/patches/gperftools-Implemented-O-log-n-searching-among-large-spans.patch
----------------------------------------------------------------------
diff --git a/thirdparty/patches/gperftools-Implemented-O-log-n-searching-among-large-spans.patch
b/thirdparty/patches/gperftools-Implemented-O-log-n-searching-among-large-spans.patch
new file mode 100644
index 0000000..87ab134
--- /dev/null
+++ b/thirdparty/patches/gperftools-Implemented-O-log-n-searching-among-large-spans.patch
@@ -0,0 +1,494 @@
+From 071cadad7a752c5678c18174a74b82858404f762 Mon Sep 17 00:00:00 2001
+From: Aliaksey Kandratsenka <alkondratenko@gmail.com>
+Date: Sat, 3 Jun 2017 15:31:06 -0700
+Subject: [PATCH 1/1] Implemented O(log n) searching among large spans
+
+This is implemented via std::set with custom STL allocator that
+delegates to PageHeapAllocator. Free large spans are not linked
+together via linked list, but inserted into std::set. Spans also store
+iterators to std::set positions pointing to them. So that removing
+span from set is fast too.
+
+Patch implemented by Aliaksey Kandratsenka and Todd Lipcon based on
+earlier research and experimentation by James Golick.
+
+Addresses issue #535
+---
+ src/page_heap.cc               |  152 +++++++++++++++++++++++++---------------
+ src/page_heap.h                |   18 ++++--
+ src/page_heap_allocator.h      |   64 +++++++++++++++++
+ src/span.h                     |   61 ++++++++++++++++-
+ src/tests/tcmalloc_unittest.cc |    2 +-
+ 5 files changed, 234 insertions(+), 63 deletions(-)
+
+diff --git a/src/page_heap.cc b/src/page_heap.cc
+index 0dda58f..712732a 100644
+--- a/src/page_heap.cc
++++ b/src/page_heap.cc
+@@ -69,8 +69,6 @@ PageHeap::PageHeap()
+       release_index_(kMaxPages),
+       aggressive_decommit_(false) {
+   COMPILE_ASSERT(kClassSizesMax <= (1 << PageMapCache::kValuebits), valuebits);
+-  DLL_Init(&large_.normal);
+-  DLL_Init(&large_.returned);
+   for (int i = 0; i < kMaxPages; i++) {
+     DLL_Init(&free_[i].normal);
+     DLL_Init(&free_[i].returned);
+@@ -168,45 +166,38 @@ Span* PageHeap::New(Length n) {
+ }
+ 
+ Span* PageHeap::AllocLarge(Length n) {
+-  // find the best span (closest to n in size).
+-  // The following loops implements address-ordered best-fit.
+   Span *best = NULL;
++  Span *best_normal = NULL;
+ 
+-  // Search through normal list
+-  for (Span* span = large_.normal.next;
+-       span != &large_.normal;
+-       span = span->next) {
+-    if (span->length >= n) {
+-      if ((best == NULL)
+-          || (span->length < best->length)
+-          || ((span->length == best->length) && (span->start < best->start)))
{
+-        best = span;
+-        ASSERT(best->location == Span::ON_NORMAL_FREELIST);
+-      }
+-    }
+-  }
++  // Create a Span to use as an upper bound.
++  Span bound;
++  bound.start = 0;
++  bound.length = n;
+ 
+-  Span *bestNormal = best;
++  // First search the NORMAL spans..
++  SpanSet::iterator place = large_normal_.upper_bound(SpanPtrWithLength(&bound));
++  if (place != large_normal_.end()) {
++    best = place->span;
++    best_normal = best;
++    ASSERT(best->location == Span::ON_NORMAL_FREELIST);
++  }
+ 
+-  // Search through released list in case it has a better fit
+-  for (Span* span = large_.returned.next;
+-       span != &large_.returned;
+-       span = span->next) {
+-    if (span->length >= n) {
+-      if ((best == NULL)
+-          || (span->length < best->length)
+-          || ((span->length == best->length) && (span->start < best->start)))
{
+-        best = span;
+-        ASSERT(best->location == Span::ON_RETURNED_FREELIST);
+-      }
+-    }
++  // Try to find better fit from RETURNED spans.
++  place = large_returned_.upper_bound(SpanPtrWithLength(&bound));
++  if (place != large_returned_.end()) {
++    Span *c = place->span;
++    ASSERT(c->location == Span::ON_RETURNED_FREELIST);
++    if (best_normal == NULL
++        || c->length < best->length
++        || (c->length == best->length && c->start < best->start))
++      best = place->span;
+   }
+ 
+-  if (best == bestNormal) {
++  if (best == best_normal) {
+     return best == NULL ? NULL : Carve(best, n);
+   }
+ 
+-  // best comes from returned list.
++  // best comes from RETURNED set.
+ 
+   if (EnsureLimit(n, false)) {
+     return Carve(best, n);
+@@ -214,13 +205,13 @@ Span* PageHeap::AllocLarge(Length n) {
+ 
+   if (EnsureLimit(n, true)) {
+     // best could have been destroyed by coalescing.
+-    // bestNormal is not a best-fit, and it could be destroyed as well.
++    // best_normal is not a best-fit, and it could be destroyed as well.
+     // We retry, the limit is already ensured:
+     return AllocLarge(n);
+   }
+ 
+-  // If bestNormal existed, EnsureLimit would succeeded:
+-  ASSERT(bestNormal == NULL);
++  // If best_normal existed, EnsureLimit would succeeded:
++  ASSERT(best_normal == NULL);
+   // We are not allowed to take best from returned list.
+   return NULL;
+ }
+@@ -406,12 +397,27 @@ void PageHeap::MergeIntoFreeList(Span* span) {
+ 
+ void PageHeap::PrependToFreeList(Span* span) {
+   ASSERT(span->location != Span::IN_USE);
+-  SpanList* list = (span->length < kMaxPages) ? &free_[span->length] : &large_;
+-  if (span->location == Span::ON_NORMAL_FREELIST) {
++  if (span->location == Span::ON_NORMAL_FREELIST)
+     stats_.free_bytes += (span->length << kPageShift);
++  else
++    stats_.unmapped_bytes += (span->length << kPageShift);
++
++  if (span->length >= kMaxPages) {
++    SpanSet *set = &large_normal_;
++    if (span->location == Span::ON_RETURNED_FREELIST)
++      set = &large_returned_;
++    std::pair<SpanSet::iterator, bool> p =
++        set->insert(SpanPtrWithLength(span));
++    ASSERT(p.second); // We never have duplicates since span->start is unique.
++    span->rev_ptr.set_iterator(p.first);
++    ASSERT(span->rev_ptr.get_iterator()->span == span);
++    return;
++  }
++
++  SpanList* list = &free_[span->length];
++  if (span->location == Span::ON_NORMAL_FREELIST) {
+     DLL_Prepend(&list->normal, span);
+   } else {
+-    stats_.unmapped_bytes += (span->length << kPageShift);
+     DLL_Prepend(&list->returned, span);
+   }
+ }
+@@ -423,7 +429,16 @@ void PageHeap::RemoveFromFreeList(Span* span) {
+   } else {
+     stats_.unmapped_bytes -= (span->length << kPageShift);
+   }
+-  DLL_Remove(span);
++  if (span->length >= kMaxPages) {
++    SpanSet *set = &large_normal_;
++    if (span->location == Span::ON_RETURNED_FREELIST)
++      set = &large_returned_;
++    ASSERT(span->rev_ptr.get_iterator()->span == span);
++    ASSERT(set->find(SpanPtrWithLength(span)) == span->rev_ptr.get_iterator());
++    set->erase(span->rev_ptr.get_iterator());
++  } else {
++    DLL_Remove(span);
++  }
+ }
+ 
+ void PageHeap::IncrementalScavenge(Length n) {
+@@ -459,8 +474,7 @@ void PageHeap::IncrementalScavenge(Length n) {
+   }
+ }
+ 
+-Length PageHeap::ReleaseLastNormalSpan(SpanList* slist) {
+-  Span* s = slist->normal.prev;
++Length PageHeap::ReleaseSpan(Span* s) {
+   ASSERT(s->location == Span::ON_NORMAL_FREELIST);
+ 
+   if (DecommitSpan(s)) {
+@@ -477,21 +491,35 @@ Length PageHeap::ReleaseLastNormalSpan(SpanList* slist) {
+ Length PageHeap::ReleaseAtLeastNPages(Length num_pages) {
+   Length released_pages = 0;
+ 
+-  // Round robin through the lists of free spans, releasing the last
+-  // span in each list.  Stop after releasing at least num_pages
++  // Round robin through the lists of free spans, releasing a
++  // span from each list.  Stop after releasing at least num_pages
+   // or when there is nothing more to release.
+   while (released_pages < num_pages && stats_.free_bytes > 0) {
+     for (int i = 0; i < kMaxPages+1 && released_pages < num_pages;
+          i++, release_index_++) {
++      Span *s;
+       if (release_index_ > kMaxPages) release_index_ = 0;
+-      SpanList* slist = (release_index_ == kMaxPages) ?
+-          &large_ : &free_[release_index_];
+-      if (!DLL_IsEmpty(&slist->normal)) {
+-        Length released_len = ReleaseLastNormalSpan(slist);
+-        // Some systems do not support release
+-        if (released_len == 0) return released_pages;
+-        released_pages += released_len;
++
++      if (release_index_ == kMaxPages) {
++        if (large_normal_.empty()) {
++          continue;
++        }
++        s = (large_normal_.begin())->span;
++      } else {
++        SpanList* slist = &free_[release_index_];
++        if (DLL_IsEmpty(&slist->normal)) {
++          continue;
++        }
++        s = slist->normal.prev;
+       }
++      // TODO(todd) if the remaining number of pages to release
++      // is significantly smaller than s->length, and s is on the
++      // large freelist, should we carve s instead of releasing?
++      // the whole thing?
++      Length released_len = ReleaseSpan(s);
++      // Some systems do not support release
++      if (released_len == 0) return released_pages;
++      released_pages += released_len;
+     }
+   }
+   return released_pages;
+@@ -544,12 +572,12 @@ void PageHeap::GetLargeSpanStats(LargeSpanStats* result) {
+   result->spans = 0;
+   result->normal_pages = 0;
+   result->returned_pages = 0;
+-  for (Span* s = large_.normal.next; s != &large_.normal; s = s->next) {
+-    result->normal_pages += s->length;;
++  for (SpanSet::iterator it = large_normal_.begin(); it != large_normal_.end(); ++it) {
++    result->normal_pages += it->length;
+     result->spans++;
+   }
+-  for (Span* s = large_.returned.next; s != &large_.returned; s = s->next) {
+-    result->returned_pages += s->length;
++  for (SpanSet::iterator it = large_returned_.begin(); it != large_returned_.end(); ++it)
{
++    result->returned_pages += it->length;
+     result->spans++;
+   }
+ }
+@@ -664,8 +692,8 @@ bool PageHeap::Check() {
+ 
+ bool PageHeap::CheckExpensive() {
+   bool result = Check();
+-  CheckList(&large_.normal, kMaxPages, 1000000000, Span::ON_NORMAL_FREELIST);
+-  CheckList(&large_.returned, kMaxPages, 1000000000, Span::ON_RETURNED_FREELIST);
++  CheckSet(&large_normal_, kMaxPages, Span::ON_NORMAL_FREELIST);
++  CheckSet(&large_returned_, kMaxPages, Span::ON_RETURNED_FREELIST);
+   for (Length s = 1; s < kMaxPages; s++) {
+     CheckList(&free_[s].normal, s, s, Span::ON_NORMAL_FREELIST);
+     CheckList(&free_[s].returned, s, s, Span::ON_RETURNED_FREELIST);
+@@ -685,4 +713,16 @@ bool PageHeap::CheckList(Span* list, Length min_pages, Length max_pages,
+   return true;
+ }
+ 
++bool PageHeap::CheckSet(SpanSet* spanset, Length min_pages,int freelist) {
++  for (SpanSet::iterator it = spanset->begin(); it != spanset->end(); ++it) {
++    Span* s = it->span;
++    CHECK_CONDITION(s->length == it->length);
++    CHECK_CONDITION(s->location == freelist);  // NORMAL or RETURNED
++    CHECK_CONDITION(s->length >= min_pages);
++    CHECK_CONDITION(GetDescriptor(s->start) == s);
++    CHECK_CONDITION(GetDescriptor(s->start+s->length-1) == s);
++  }
++  return true;
++}
++
+ }  // namespace tcmalloc
+diff --git a/src/page_heap.h b/src/page_heap.h
+index b688512..34f18bb 100644
+--- a/src/page_heap.h
++++ b/src/page_heap.h
+@@ -196,6 +196,7 @@ class PERFTOOLS_DLL_DECL PageHeap {
+   bool CheckExpensive();
+   bool CheckList(Span* list, Length min_pages, Length max_pages,
+                  int freelist);  // ON_NORMAL_FREELIST or ON_RETURNED_FREELIST
++  bool CheckSet(SpanSet *s, Length min_pages, int freelist);
+ 
+   // Try to release at least num_pages for reuse by the OS.  Returns
+   // the actual number of pages released, which may be less than
+@@ -264,8 +265,12 @@ class PERFTOOLS_DLL_DECL PageHeap {
+     Span        returned;
+   };
+ 
+-  // List of free spans of length >= kMaxPages
+-  SpanList large_;
++  // Sets of spans with length >= kMaxPages.
++  //
++  // Rather than using a linked list, we use sets here for efficient
++  // best-fit search.
++  SpanSet large_normal_;
++  SpanSet large_returned_;
+ 
+   // Array mapping from span length to a doubly linked list of free spans
+   SpanList free_[kMaxPages];
+@@ -317,9 +322,12 @@ class PERFTOOLS_DLL_DECL PageHeap {
+   // IncrementalScavenge(n) is called whenever n pages are freed.
+   void IncrementalScavenge(Length n);
+ 
+-  // Release the last span on the normal portion of this list.
+-  // Return the length of that span or zero if release failed.
+-  Length ReleaseLastNormalSpan(SpanList* slist);
++  // Attempts to decommit 's' and move it to the returned freelist.
++  //
++  // Returns the length of the Span or zero if release failed.
++  //
++  // REQUIRES: 's' must be on the NORMAL freelist.
++  Length ReleaseSpan(Span *s);
+ 
+   // Checks if we are allowed to take more memory from the system.
+   // If limit is reached and allowRelease is true, tries to release
+diff --git a/src/page_heap_allocator.h b/src/page_heap_allocator.h
+index 892d1c1..440b8ad 100644
+--- a/src/page_heap_allocator.h
++++ b/src/page_heap_allocator.h
+@@ -109,6 +109,70 @@ class PageHeapAllocator {
+   int inuse_;
+ };
+ 
++// STL-compatible allocator which forwards allocations to a PageHeapAllocator.
++//
++// Like PageHeapAllocator, this requires external synchronization. To avoid multiple
++// separate STLPageHeapAllocator<T> from sharing the same underlying PageHeapAllocator<T>,
++// the |LockingTag| template argument should be used. Template instantiations with
++// different locking tags can safely be used concurrently.
++template <typename T, class LockingTag>
++class STLPageHeapAllocator {
++ public:
++  typedef size_t     size_type;
++  typedef ptrdiff_t  difference_type;
++  typedef T*         pointer;
++  typedef const T*   const_pointer;
++  typedef T&         reference;
++  typedef const T&   const_reference;
++  typedef T          value_type;
++
++  template <class T1> struct rebind {
++    typedef STLPageHeapAllocator<T1, LockingTag> other;
++  };
++
++  STLPageHeapAllocator() { }
++  STLPageHeapAllocator(const STLPageHeapAllocator&) { }
++  template <class T1> STLPageHeapAllocator(const STLPageHeapAllocator<T1, LockingTag>&)
{ }
++  ~STLPageHeapAllocator() { }
++
++  pointer address(reference x) const { return &x; }
++  const_pointer address(const_reference x) const { return &x; }
++
++  size_type max_size() const { return size_t(-1) / sizeof(T); }
++
++  void construct(pointer p, const T& val) { ::new(p) T(val); }
++  void construct(pointer p) { ::new(p) T(); }
++  void destroy(pointer p) { p->~T(); }
++
++  // There's no state, so these allocators are always equal
++  bool operator==(const STLPageHeapAllocator&) const { return true; }
++
++  pointer allocate(size_type n, const void* = 0) {
++    if (!underlying_.initialized) {
++      underlying_.allocator.Init();
++      underlying_.initialized = true;
++    }
++
++    CHECK_CONDITION(n == 1);
++    return underlying_.allocator.New();
++  }
++  void deallocate(pointer p, size_type n) {
++    CHECK_CONDITION(n == 1);
++    underlying_.allocator.Delete(p);
++  }
++
++ private:
++  struct Storage {
++    explicit Storage(base::LinkerInitialized x) {}
++    PageHeapAllocator<T> allocator;
++    bool initialized;
++  };
++  static Storage underlying_;
++};
++
++template<typename T, class LockingTag>
++typename STLPageHeapAllocator<T, LockingTag>::Storage STLPageHeapAllocator<T, LockingTag>::underlying_(base::LINKER_INITIALIZED);
++
+ }  // namespace tcmalloc
+ 
+ #endif  // TCMALLOC_PAGE_HEAP_ALLOCATOR_H_
+diff --git a/src/span.h b/src/span.h
+index 83feda1..4e343ba 100644
+--- a/src/span.h
++++ b/src/span.h
+@@ -37,17 +37,63 @@
+ #define TCMALLOC_SPAN_H_
+ 
+ #include <config.h>
++#include <set>
+ #include "common.h"
++#include "base/logging.h"
++#include "page_heap_allocator.h"
+ 
+ namespace tcmalloc {
+ 
++struct SpanBestFitLess;
++struct Span;
++
++// Store a pointer to a span along with a cached copy of its length.
++// These are used as set elements to improve the performance of
++// comparisons during tree traversal: the lengths are inline with the
++// tree nodes and thus avoid expensive cache misses to dereference
++// the actual Span objects in most cases.
++struct SpanPtrWithLength {
++  explicit SpanPtrWithLength(Span* s);
++
++  Span* span;
++  Length length;
++};
++typedef std::set<SpanPtrWithLength, SpanBestFitLess, STLPageHeapAllocator<Span*, void>
> SpanSet;
++
++// Comparator for best-fit search, with address order as a tie-breaker.
++struct SpanBestFitLess {
++  bool operator()(SpanPtrWithLength a, SpanPtrWithLength b);
++};
++
++// Wrapper which stores a SpanSet::iterator as a POD type.
++// This allows the iterator to be stored in a union below.
++struct SpanSetRevPtr {
++  char data[sizeof(SpanSet::iterator)];
++
++  SpanSet::iterator get_iterator() {
++    SpanSet::iterator ret;
++    memcpy(&ret, this, sizeof(ret));
++    return ret;
++  }
++
++  void set_iterator(const SpanSet::iterator& val) {
++    new (this) SpanSet::iterator(val);
++  }
++};
++
+ // Information kept for a span (a contiguous run of pages).
+ struct Span {
++  Span() {}
+   PageID        start;          // Starting page number
+   Length        length;         // Number of pages in span
+   Span*         next;           // Used when in link list
+   Span*         prev;           // Used when in link list
+-  void*         objects;        // Linked list of free objects
++  union {
++    void*         objects;      // Linked list of free objects
++    SpanSetRevPtr rev_ptr;      // "pointer" (std::set iterator) to
++                                // SpanSet entry associated with this
++                                // Span.
++  };
+   unsigned int  refcount : 16;  // Number of non-free objects
+   unsigned int  sizeclass : 8;  // Size-class for small objects (or 0)
+   unsigned int  location : 2;   // Is the span on a freelist, and if so, which?
+@@ -71,6 +117,19 @@ void Event(Span* span, char op, int v = 0);
+ #define Event(s,o,v) ((void) 0)
+ #endif
+ 
++inline SpanPtrWithLength::SpanPtrWithLength(Span* s)
++    : span(s),
++      length(s->length) {
++}
++
++inline bool SpanBestFitLess::operator()(SpanPtrWithLength a, SpanPtrWithLength b) {
++  if (a.length < b.length)
++    return true;
++  if (a.length > b.length)
++    return false;
++  return a.span->start < b.span->start;
++}
++
+ // Allocator/deallocator for spans
+ Span* NewSpan(PageID p, Length len);
+ void DeleteSpan(Span* span);
+diff --git a/src/tests/tcmalloc_unittest.cc b/src/tests/tcmalloc_unittest.cc
+index 25b2e41..a9c6429 100644
+--- a/src/tests/tcmalloc_unittest.cc
++++ b/src/tests/tcmalloc_unittest.cc
+@@ -1422,7 +1422,7 @@ static int RunAllTests(int argc, char** argv) {
+ 
+     // Try strdup(), which the system allocates but we must free.  If
+     // all goes well, libc will use our malloc!
+-    p2 = strdup("test");
++    p2 = strdup("in memory of James Golick");
+     CHECK(p2 != NULL);
+     VerifyNewHookWasCalled();
+     free(p2);
+-- 
+1.7.1
+

http://git-wip-us.apache.org/repos/asf/kudu/blob/1ae56048/thirdparty/patches/gperftools-Replace-namespace-base-with-namespace-tcmalloc.patch
----------------------------------------------------------------------
diff --git a/thirdparty/patches/gperftools-Replace-namespace-base-with-namespace-tcmalloc.patch
b/thirdparty/patches/gperftools-Replace-namespace-base-with-namespace-tcmalloc.patch
index 8660410..eca38d7 100644
--- a/thirdparty/patches/gperftools-Replace-namespace-base-with-namespace-tcmalloc.patch
+++ b/thirdparty/patches/gperftools-Replace-namespace-base-with-namespace-tcmalloc.patch
@@ -1,49 +1,52 @@
-From c63fd16547503fb9d9b0a3e35e16a1a66fbfeaa0 Mon Sep 17 00:00:00 2001
+From 181843b0ecddcc8c8be912cf3c035e27e2f63842 Mon Sep 17 00:00:00 2001
 From: Todd Lipcon <todd@cloudera.com>
-Date: Tue, 23 Jan 2018 11:25:50 -0800
-Subject: [PATCH 1/1] Replace namespace base with namespace tcmalloc
+Date: Mon, 5 Feb 2018 21:19:54 -0800
+Subject: [PATCH 3/3] Substitute namespaces
 
+perl -p -i -e 's,base::,tcmalloc::,g' $(find . -name *.h -o -name \*.cc)
+perl -p -i -e 's,namespace base,namespace tcmalloc,g' $(find . -name *.h -o -name \*.cc)
 ---
- src/base/atomicops-internals-arm-generic.h |  6 +-
- src/base/atomicops-internals-arm-v6plus.h  |  4 +-
- src/base/atomicops-internals-gcc.h         |  6 +-
- src/base/atomicops-internals-linuxppc.h    |  6 +-
- src/base/atomicops-internals-macosx.h      |  8 +--
- src/base/atomicops-internals-mips.h        |  6 +-
- src/base/atomicops-internals-windows.h     | 14 ++---
- src/base/atomicops-internals-x86.h         |  6 +-
- src/base/atomicops.h                       | 92 +++++++++++++++---------------
- src/base/basictypes.h                      |  8 +--
- src/base/elf_mem_image.cc                  |  4 +-
- src/base/elf_mem_image.h                   |  4 +-
- src/base/spinlock.cc                       | 16 +++---
- src/base/spinlock.h                        | 14 ++---
- src/base/spinlock_internal.cc              | 18 +++---
- src/base/spinlock_internal.h               |  4 +-
- src/base/spinlock_linux-inl.h              |  6 +-
- src/base/spinlock_posix-inl.h              |  6 +-
- src/base/spinlock_win32-inl.h              |  6 +-
- src/base/vdso_support.cc                   |  4 +-
- src/base/vdso_support.h                    |  4 +-
- src/central_freelist.h                     |  2 +-
- src/emergency_malloc.cc                    |  2 +-
- src/gperftools/malloc_extension.h          |  8 +--
- src/heap-profile-table.cc                  |  2 +-
- src/internal_logging.cc                    |  2 +-
- src/malloc_hook-inl.h                      | 44 +++++++-------
- src/malloc_hook.cc                         | 60 +++++++++----------
- src/page_heap.cc                           | 10 ++--
- src/page_heap.h                            |  4 +-
- src/raw_printer.cc                         |  2 +-
- src/raw_printer.h                          |  2 +-
- src/stacktrace_powerpc-linux-inl.h         |  4 +-
- src/stacktrace_x86-inl.h                   |  6 +-
- src/tcmalloc.cc                            | 10 ++--
- src/tests/atomicops_unittest.cc            | 36 ++++++------
- src/tests/malloc_hook_test.cc              |  4 +-
- src/tests/raw_printer_test.cc              |  8 +--
- src/tests/tcmalloc_unittest.cc             | 28 ++++-----
- 39 files changed, 238 insertions(+), 238 deletions(-)
+ src/base/atomicops-internals-arm-generic.h |    6 +-
+ src/base/atomicops-internals-arm-v6plus.h  |    4 +-
+ src/base/atomicops-internals-gcc.h         |    6 +-
+ src/base/atomicops-internals-linuxppc.h    |    6 +-
+ src/base/atomicops-internals-macosx.h      |    8 +-
+ src/base/atomicops-internals-mips.h        |    6 +-
+ src/base/atomicops-internals-windows.h     |   14 ++--
+ src/base/atomicops-internals-x86.h         |    6 +-
+ src/base/atomicops.h                       |   92 ++++++++++++++--------------
+ src/base/basictypes.h                      |    8 +-
+ src/base/elf_mem_image.cc                  |    4 +-
+ src/base/elf_mem_image.h                   |    4 +-
+ src/base/spinlock.cc                       |   16 +++---
+ src/base/spinlock.h                        |   14 ++--
+ src/base/spinlock_internal.cc              |   18 +++---
+ src/base/spinlock_internal.h               |    4 +-
+ src/base/spinlock_linux-inl.h              |    6 +-
+ src/base/spinlock_posix-inl.h              |    6 +-
+ src/base/spinlock_win32-inl.h              |    6 +-
+ src/base/vdso_support.cc                   |    4 +-
+ src/base/vdso_support.h                    |    4 +-
+ src/central_freelist.h                     |    2 +-
+ src/emergency_malloc.cc                    |    2 +-
+ src/gperftools/malloc_extension.h          |    8 +-
+ src/heap-profile-table.cc                  |    2 +-
+ src/internal_logging.cc                    |    2 +-
+ src/malloc_hook-inl.h                      |   44 +++++++-------
+ src/malloc_hook.cc                         |   60 +++++++++---------
+ src/page_heap.cc                           |   10 ++--
+ src/page_heap.h                            |    4 +-
+ src/page_heap_allocator.h                  |    4 +-
+ src/raw_printer.cc                         |    2 +-
+ src/raw_printer.h                          |    2 +-
+ src/stacktrace_powerpc-linux-inl.h         |    4 +-
+ src/stacktrace_x86-inl.h                   |    6 +-
+ src/tcmalloc.cc                            |   10 ++--
+ src/tests/atomicops_unittest.cc            |   36 ++++++------
+ src/tests/malloc_hook_test.cc              |    4 +-
+ src/tests/raw_printer_test.cc              |    8 +-
+ src/tests/tcmalloc_unittest.cc             |   28 ++++----
+ 40 files changed, 240 insertions(+), 240 deletions(-)
 
 diff --git a/src/base/atomicops-internals-arm-generic.h b/src/base/atomicops-internals-arm-generic.h
 index d0f9413..c81f1e6 100644
@@ -1296,10 +1299,10 @@ index 64c2165..1b0dcd0 100644
  // These are available as C bindings as well as C++, hence their
  // definition outside the MallocHook class.
 diff --git a/src/page_heap.cc b/src/page_heap.cc
-index 0dda58f..fa16001 100644
+index 712732a..c1c7ed3 100644
 --- a/src/page_heap.cc
 +++ b/src/page_heap.cc
-@@ -554,7 +554,7 @@ void PageHeap::GetLargeSpanStats(LargeSpanStats* result) {
+@@ -582,7 +582,7 @@ void PageHeap::GetLargeSpanStats(LargeSpanStats* result) {
    }
  }
  
@@ -1308,7 +1311,7 @@ index 0dda58f..fa16001 100644
    Span* span = reinterpret_cast<Span*>(pagemap_.Next(start));
    if (span == NULL) {
      return false;
-@@ -564,7 +564,7 @@ bool PageHeap::GetNextRange(PageID start, base::MallocRange* r) {
+@@ -592,7 +592,7 @@ bool PageHeap::GetNextRange(PageID start, base::MallocRange* r) {
    r->fraction = 0;
    switch (span->location) {
      case Span::IN_USE:
@@ -1317,7 +1320,7 @@ index 0dda58f..fa16001 100644
        r->fraction = 1;
        if (span->sizeclass > 0) {
          // Only some of the objects in this span may be in use.
-@@ -573,13 +573,13 @@ bool PageHeap::GetNextRange(PageID start, base::MallocRange* r) {
+@@ -601,13 +601,13 @@ bool PageHeap::GetNextRange(PageID start, base::MallocRange* r) {
        }
        break;
      case Span::ON_NORMAL_FREELIST:
@@ -1335,7 +1338,7 @@ index 0dda58f..fa16001 100644
    }
    return true;
 diff --git a/src/page_heap.h b/src/page_heap.h
-index b688512..91b7587 100644
+index 34f18bb..c040b2d 100644
 --- a/src/page_heap.h
 +++ b/src/page_heap.h
 @@ -65,7 +65,7 @@
@@ -1356,6 +1359,28 @@ index b688512..91b7587 100644
  
    // Page heap statistics
    struct Stats {
+diff --git a/src/page_heap_allocator.h b/src/page_heap_allocator.h
+index 440b8ad..bc60a21 100644
+--- a/src/page_heap_allocator.h
++++ b/src/page_heap_allocator.h
+@@ -163,7 +163,7 @@ class STLPageHeapAllocator {
+ 
+  private:
+   struct Storage {
+-    explicit Storage(base::LinkerInitialized x) {}
++    explicit Storage(tcmalloc::LinkerInitialized x) {}
+     PageHeapAllocator<T> allocator;
+     bool initialized;
+   };
+@@ -171,7 +171,7 @@ class STLPageHeapAllocator {
+ };
+ 
+ template<typename T, class LockingTag>
+-typename STLPageHeapAllocator<T, LockingTag>::Storage STLPageHeapAllocator<T, LockingTag>::underlying_(base::LINKER_INITIALIZED);
++typename STLPageHeapAllocator<T, LockingTag>::Storage STLPageHeapAllocator<T, LockingTag>::underlying_(tcmalloc::LINKER_INITIALIZED);
+ 
+ }  // namespace tcmalloc
+ 
 diff --git a/src/raw_printer.cc b/src/raw_printer.cc
 index 3cf028e..cdcf8a1 100644
 --- a/src/raw_printer.cc
@@ -1597,7 +1622,7 @@ index 2c7be6a..99a2f13 100644
    CHECK_EQ(string("hello world"), string(buffer));
    CHECK_EQ(11, printer.length());
 diff --git a/src/tests/tcmalloc_unittest.cc b/src/tests/tcmalloc_unittest.cc
-index 25b2e41..5b0a906 100644
+index a9c6429..9149fde 100644
 --- a/src/tests/tcmalloc_unittest.cc
 +++ b/src/tests/tcmalloc_unittest.cc
 @@ -853,21 +853,21 @@ namespace {
@@ -1666,5 +1691,5 @@ index 25b2e41..5b0a906 100644
  
  #ifndef DEBUGALLOCATION
 -- 
-1.8.3.1
+1.7.1
 


Mime
View raw message