aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Farner <wfar...@apache.org>
Subject Re: Non-exclusive dedicated constraint
Date Wed, 20 Jan 2016 03:53:08 GMT
Joe - if you want to pursue this, I suggest you start another thread to
keep this thread's discussion in tact.  I will not be able to lead this
change, but can certainly shepherd!

On Tuesday, January 19, 2016, Joe Smith <yasumoto7@gmail.com> wrote:

> As an operator, that'd be a relatively simple change in tooling, and the
> benefits of not forcing a slave restart would be _huge_.
>
> Keeping the dedicated semantics (but adding non-exclusive) would be ideal
> if possible.
>
> > On Jan 19, 2016, at 19:09, Bill Farner <wfarner@apache.org
> <javascript:;>> wrote:
> >
> > Also, regarding dedicated constraints necessitating a slave restart -
> i've
> > pondered moving dedicated machine management to the scheduler for similar
> > purposes.  There's not really much forcing that behavior to be managed
> with
> > a slave attribute.
> >
> > On Tue, Jan 19, 2016 at 7:05 PM, John Sirois <john@conductant.com
> <javascript:;>> wrote:
> >
> >> On Tue, Jan 19, 2016 at 7:22 PM, Maxim Khutornenko <maxim@apache.org
> <javascript:;>>
> >> wrote:
> >>
> >>> Has anyone explored an idea of having a non-exclusive (wrt job role)
> >>> dedicated constraint in Aurora before?
> >>
> >>
> >>> We do have a dedicated constraint now but it assumes a 1:1
> >>> relationship between a job role and a slave attribute [1]. For
> >>> example: a 'www-data/prod/hello' job with a dedicated constraint of
> >>> 'dedicated': 'www-data/hello' may only be pinned to a particular set
> >>> of slaves if all of them have 'www-data/hello' attribute set. No other
> >>> role tasks will be able to land on those slaves unless their
> >>> 'role/name' pair is added into the slave attribute set.
> >>>
> >>> The above is very limiting as it prevents carving out subsets of a
> >>> shared pool cluster to be used by multiple roles at the same time.
> >>> Would it make sense to have a free-form dedicated constraint not bound
> >>> to a particular role? Multiple jobs could then use this type of
> >>> constraint dynamically without modifying the slave command line (and
> >>> requiring slave restart).
> >>
> >> Can't this just be any old Constraint (not named "dedicated").  In other
> >> words, doesn't this code already deal with non-dedicated constraints?:
> >>
> >>
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java#L193-L197
> >>
> >>
> >>> This could be quite useful for experimenting purposes (e.g. different
> >>> host OS) or to target a different hardware offering (e.g. GPUs). In
> >>> other words, only those jobs that explicitly opt-in to participate in
> >>> an experiment or hw offering would be landing on that slave set.
> >>>
> >>> Thanks,
> >>> Maxim
> >>>
> >>> [1]-
> >>
> https://github.com/apache/aurora/blob/eec985d948f02f46637d87cd4d212eb2a70ef8d0/src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java#L272-L276
> >>
> >>
> >>
> >> --
> >> John Sirois
> >> 303-512-3301
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message