mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Rojas <alexan...@mesosphere.io>
Subject Re: [Proposal] Use jemalloc as default memory allocator for Mesos
Date Mon, 21 Aug 2017 14:26:57 GMT
Hi Benno,

This does sound like a great addition to Mesos. Can you however explain how jemalloc is better
than tcmalloc? I think that for such important change, we probably need some more information.

Your comment in MESOS-7876 mentions that we already have tcmalloc since it is part of gperftools,
so I would like to have a whole picture of the advantages and disadvantages of both options.

Alexander Rojas
alexander@mesosphere.io




> On 18. Aug 2017, at 12:49, Benno Evers <bevers@mesosphere.com> wrote:
> 
> Hi all,
> 
> I would like to propose bundling jemalloc as a new dependency
> under `3rdparty/`, and to link Mesos against this new memory
> allocator by default.
> 
> 
> # Motivation
> 
> The Mesos master and agent binaries are, ideally, very long-running
> processes. This makes them susceptible to memory issues, because
> even small leaks have a chance to build up over time to the point
> where they become problematic.
> 
> We have seen several such issues on our internal Mesos installations,
> for example https://issues.apache.org/jira/browse/MESOS-7748
> or https://issues.apache.org/jira/browse/MESOS-7800.
> 
> I imagine any organization running Mesos for an extended period
> of time has had its share of similar issues, so I expect this
> proposal to be useful for the whole community.
> 
> 
> # Why jemalloc?
> 
> Given that memory issues tend to be most visible after a given
> process has been running for a long time, it would be great to
> have the option to enable heap tracking and profiling at runtime,
> without having to restart the process. (This ability could then
> be connected to a Mesos endpoint, similar to how we can adjust
> the log level at runtime)
> 
> The two production-quality memory allocators that have this
> ability currently seem to be tcmalloc and jemalloc. Of these,
> jemalloc does produce in our experience better and more
> detailed statistics.
> 
> 
> # What is the impact on users who do not need this feature?
> 
> Naturally, not every single user of Mesos will have a need
> for this feature. To ensure these users would not experience serious
> performance regressions as a result of this change, we conducted
> a preliminary set of benchmarks whose results are collected
> under https://issues.apache.org/jira/browse/MESOS-7876
> 
> It turns out that we could probably even expect a small speedup (1% - 5%)
> as a nice side-effect of this change.
> 
> Users who compile Mesos themselves would of course have the option
> to disable jemalloc at configuration time or replace it with their
> memory allocator of choice.
> 
> 
> 
> I'm looking forward to hear any thoughts and comments.
> 
> 
> Thanks,
> -- 
> Benno Evers
> Software Engineer, Mesosphere


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message