mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Massenzio <ma...@mesosphere.io>
Subject Re: Configuration file?
Date Tue, 24 Nov 2015 16:48:58 GMT
Thank you for asking :)

The idea is pretty simple (and not even original, it's the "Facade
pattern"): essentially we would define a YAML syntax for the various
configuration flags that Mesos ({Master, Agent}) define, possibly grouping
them by "topic" and then write a few python classes, that read in the YAML
configuration and emit a set of flags that reflect the caller intent, then
invoke the appropriate ./bin/mesos-{master,slave}.sh script.

(variation on the theme for installed packages)

E.g.:

# Configuration /opt/mesos/config-master.yml

ip: 192.168.1.101

zk:
  url: zk://192.168.1.220:2181
  quorum: 3

directories:
  work: /var/run/mesos
  log: /var/log/mesos/master.log

Execute with:

  ./mesos-wrapper.py --run master --config /opt/mesos/config-master.yml
--no-ssl \
     [other script-specific options]

would generate something along the lines of:

  /opt/mesos/bin/mesos-master.sh --ip=192.168.1.101 \
    --zk=zk://192.168.1.220:2181 --quorum=3 \
    --work_dir=/var/run/mesos --log_dir=/var/log/mesos/master.log

(example greatly contrived and probably pointless, but hopefully gives you
the idea).

The point is that leveraging Python and the existing YAML libraries, this
would be pretty quick to implement, would leave Mesos per se completely
untouched, and would be easy to maintain (add/remove flags as needed;
enforce deprecation cycles; etc.)

There's probably a ton of detail I'm missing, but you get the point.

Cheers,
M.

On Tuesday, November 24, 2015, tommy xiao <xiaods@gmail.com> wrote:

> how to do that? Marco
>
> 2015-11-24 3:54 GMT+08:00 Marco Massenzio <marco@mesosphere.io
> <javascript:;>>:
>
> > I was thinking along the same lines, with a slightly more "modern"
> approach
> > :)
> >
> > My idea was to write a thin Python layer that reads a configuration YAML
> > file (possibly with "override" flags) and then invokes
> mesos-{master,slave}
> > with the appropriate flags.
> >
> > By using YAML, we would also gain the ability to have more intuitive
> > syntax; grouping flags by function; and make it extensible.
> >
> > Any takers who may want to work together on this one?
> >
> > --
> > *Marco Massenzio*
> > Distributed Systems Engineer
> > http://codetrips.com
> >
> > On Mon, Nov 23, 2015 at 11:18 AM, Vinod Kone <vinodkone@gmail.com
> <javascript:;>> wrote:
> >
> > > We had this discussion a long while ago and the argument was that we
> > > already have a configuration file of sorts. It is a bash script with
> one
> > > flag per line that sets the environment. Is this not sufficient?
> > >
> > > Example:
> > >
> > > config.sh
> > > ------------
> > > MESOS_WORK_DIR="/var/run/mesos"
> > > MESOS_QUORUM=2
> > >
> > > $ source config.sh
> > > $ ./bin/mesos-master.sh
> > >
> > >
> > >
> > > On Mon, Nov 23, 2015 at 10:19 AM, Jojy Varghese <jojy@mesosphere.io
> <javascript:;>>
> > > wrote:
> > >
> > > > Thanks for bringing this topic up Alex. I have been thinking about
> the
> > > same
> > > > and was wondering if we can have subsystem specific flags and some
> > scheme
> > > > where there can be a namespace instead of a flat space. And as alex
> > > > suggested, a configuration to represent this?
> > > >
> > > > -jojy
> > > > On Mon, Nov 23, 2015 at 9:32 AM tommy xiao <xiaods@gmail.com
> <javascript:;>> wrote:
> > > >
> > > > > please file a issue to tracking this proposal
> > > > >
> > > > > 2015-11-23 21:53 GMT+08:00 Klaus Ma <klaus1982.cn@gmail.com
> <javascript:;>>:
> > > > >
> > > > > > +1, that's helpful :).
> > > > > >
> > > > > > For the detail of implementing such as auto re-load, I think
we
> can
> > > let
> > > > > > owner/shepherd to decide :).
> > > > > >
> > > > > > ----
> > > > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > > > Platform Symphony/DCOS Development & Support, STG, IBM GCG
> > > > > > +86-10-8245 4084 | klaus1982.cn@gmail.com <javascript:;>
|
> http://k82.me
> > > > > >
> > > > > > On Mon, Nov 23, 2015 at 9:40 PM, Adam Avilla <adam@avil.la
> <javascript:;>> wrote:
> > > > > >
> > > > > > > +1 I think it would be helpful.
> > > > > > >
> > > > > > > This may be orthogonal / feature creep, but would it be
> possible
> > to
> > > > > have
> > > > > > > the config file be able to be safely reloaded with a HUP
or
> > > > appropriate
> > > > > > > signal?
> > > > > > >
> > > > > > > On Mon, Nov 23, 2015 at 5:32 AM, Guangya Liu <
> gyliu513@gmail.com <javascript:;>
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > +1000, introducing a new configuration file for mesos
master
> > and
> > > > > slave
> > > > > > > can
> > > > > > > > help end user take the configuration file as the source
of
> all
> > > > flags.
> > > > > > > >
> > > > > > > > The OpenStack is also using same way to manage all
of the
> > flags,
> > > it
> > > > > is
> > > > > > > > putting all flags into a configuration file and the
> > configuration
> > > > > file
> > > > > > > > including all flag examples. Most of the flags are
disabled
> by
> > > > > default
> > > > > > > and
> > > > > > > > the end user can just enable those flags based on
his
> > > requirement.
> > > > > > > >
> > > > > > > > Also the flags in the configuration file can be classified
to
> > > > > different
> > > > > > > > groups for a better management, and mesos can also
follow
> this
> > to
> > > > > > > classify
> > > > > > > > those flags to different groups, such as ACL, Cluster,
> > framework
> > > > etc.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Nov 23, 2015 at 9:08 PM, Alexander Rojas <
> > > > > > > alexander@mesosphere.io <javascript:;>>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hey guys,
> > > > > > > > >
> > > > > > > > > Over the time I’ve been involved in Mesos I’ve
seen that we
> > > went
> > > > > > from a
> > > > > > > > > handful of flags to around 42 supported flags
in the
> master.
> > At
> > > > > this
> > > > > > > > point
> > > > > > > > > I’m wondering if perhaps we should support
a configuration
> > file
> > > > in
> > > > > > > > > conjunction (or instead of) with all the command
flags.
> > > > > > > > >
> > > > > > > > > My intuition is that it will make it easier for
operators
> as
> > > well
> > > > > as
> > > > > > > for
> > > > > > > > > debuggers to be able to replicate configurations
easier.
> > > > > > > > >
> > > > > > > > > Any comments on this idea?
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > /adam
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Deshi Xiao
> > > > > Twitter: xds2000
> > > > > E-mail: xiaods(AT)gmail.com
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>


-- 
--
*Marco Massenzio*
Distributed Systems Engineer
http://codetrips.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message