mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomas Barton <barton.to...@gmail.com>
Subject Re: Recommended resources for master / scheduler machines
Date Thu, 08 Jan 2015 15:00:11 GMT
Hi Itamar,

there's definitely certain limit of machines which can Mesos master handle.
This limit is between 10 000 - 20 000 (that's number
reported by Twitter). This bottleneck is caused by event loop which handles
communication at master.

With hundreds of machines you should be fine. Only in case that your
framework scheduler would demand
too many resources for computing allocations you might encounter some
problems.

How does the strength of the master & scheduler machines affect the overall
> cluster performance?


I would say that the network is usually the main bottleneck. Adding extra
RAM won't improve mesos-master
performance. Of course if there's high CPU load on master you might observe
performance regression. Also
this depends on granularity of your tasks, if you have few long running
tasks or many short tasks (which runs
just hundreds of ms).

Tomas


On 6 January 2015 at 10:12, Itamar Ostricher <itamar@yowza3d.com> wrote:

> Are there recommendations regarding master / scheduler machines resources
> as function of cluster size?
>
> Say I have a cluster with hundreds of slave machines and thousands of
> CPUs, with a single framework that will schedule millions of tasks.
> How does the strength of the master & scheduler machines affect the
> overall cluster performance?
>
> Thanks,
> - Itamar.
>

Mime
View raw message