mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dick Davies <>
Subject Re: Thoughts and opinions in physically building a cluster
Date Thu, 25 Jun 2015 16:46:15 GMT
That doesn't sound too bad (it's a fairly typical setup e.g. on an Amazon VPC).
You probably want to avoid NAT or similar things between master and
slaves to avoid
a lot of LIBPROCESS_IP tricks so same switch sounds good.

Personally I quite like the master/slave distinction.

I wouldn't want a runaway set of  tasks to bog down the masters and
operationally we'd alert
if we're starting to lose masters whereas the slaves are 'cattle' and
we can just spin up more as
they die if need be (it's a little more tricky to scale out masters
and zookeepers so they get treated
as though they were a bit less expendable).

I co-locate the zookeeper ensemble on the masters on smaller clusters
to save VM count,
but that's more personal taste than anything.

On 25 June 2015 at 17:12, Daniel Gaston <> wrote:
> So this may be another relatively noob question, but when designing a mesos cluster,
is it basically as simple as the nodes connected by a switch? Since any of the nodes can be
"master nodes" or acting as both master and slave, I am guessing there is no need for another
head node as you would have with a traditional cluster design. But would each of the nodes
then have to be connected to the external/institutional network?
> My rough idea was for this small cluster to not be connected to the main institutional
network but for my workstation to be connected to both the cluster's network as well as to
the institutional network
> ________________________________________
> From: CCAAT <>
> Sent: June-19-15 4:57 PM
> To:
> Cc:
> Subject: Re: Thoughts and opinions in physically building a cluster
> On 06/19/2015 01:28 PM, Daniel Gaston wrote:
>> On 19/06/2015 18:38, Oliver Nicholas wrote:
>>> Unless you have some true HA requirements, it seems intuitively
>>> wasteful to have 3 masters and 2 slaves (unless the cost of 5 nodes is
>>> inconsequential to you and you hate the environment).
>> Any particular reason not to have three nodes which are acting both as
>> master and slaves?
>> None at all. I'm not a cluster or networking guru, and have only played with mesos
>> cloud-based settings so I wasn't sure how this would work. But it makes sense, that
>> the 'standby' masters are still participating in the zookeeper quorum while still
>> available to do real work as slave nodes.
> Daniel. There is no such thing as a 'cluster guru'. It's all 'seat of
> the pants' flying right now; so you are fine what you are doing and
> propose. If codes do not exist to meet your specific needs and goals,
> they can  (should?) be created.
> I'm working on an architectural expansion Where nodes (virtual, actual
> or bare metal) migrate from master --> entrepreneur --> worker --> slave
> --> embedded (bare metal or specially attached hardware. I'm proposing
> to do all of this with the "Autonomy_Function" and decisions being made
> bottom_up as opposed to the current top_down dichotomy. I'm prolly going
> to have to 'fork codes' for a while to get things stable and then
> hope they are included; when other minds see the validity of the ideas.
> Surely one "box" can be set up as both master and slave. Moving slaves
> to masters, should be an automatic function and will prolly will be
> address in the future codes of mesos.
> PS: Keep pushing your ideas and do not take no for an answer!
> Mesos belongs to everybody.....
> hth,
> James

View raw message