aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David McLaughlin <dmclaugh...@apache.org>
Subject Re: Aurora now supports multiple executors
Date Fri, 05 Aug 2016 17:26:26 GMT
Congratulations Renan on contributing such a major feature back to the
community. Looking forward to reading the blog post.


On Fri, Aug 5, 2016 at 10:15 AM, <meghdoot_b@yahoo.com.invalid> wrote:

> Expect that to be up by mid next week and put in 0.16 docs.
>
> Sent from my iPhone
>
> > On Aug 5, 2016, at 9:23 AM, Zhitao Li <zhitaoli.cs@gmail.com> wrote:
> >
> > This sounds awesome!
> >
> > Is there a document for how to use the feature?
> >
> > On Fri, Aug 5, 2016 at 12:35 AM, meghdoot bhattacharya <
> > meghdoot_b@yahoo.com.invalid> wrote:
> >
> >> Renaming the subject.
> >> Renan's latest patch in 0.16 has completed the goal of implementing
> both 2
> >> and 3. So, Aurora now can not only support custom executors but run
> >> multiple executors in the cluster!! Mesos fetcher has been integrated as
> >> well to help the custom executors have access to downloaded artifacts
> that
> >> it needs. We are going to add dedicated documentation on how to enable
> this
> >> in 0.16.
> >> This was 2 summers in making with Renan's internship slots. Special
> thanks
> >> to Maxim, Joshua and Bill to make this happen and mentoring Renan in the
> >> process.
> >>
> >>
> >> As for point 1, we are going to put out the golang client in few weeks
> to
> >> easily test different executors (including thermos) with Aurora with
> real
> >> examples. During the same time we will release production grade
> >> docker-compose executor simulating running container pods with Aurora.
> >> Expect a detailed blog in a month.
> >> Thx
> >>      From: David McLaughlin <dmclaughlin@apache.org>
> >> To: dev@aurora.apache.org
> >> Sent: Monday, June 13, 2016 10:39 AM
> >> Subject: Re: Golang Aurora lib, multiple executor support, integrate
> >> mesos task related fields
> >>
> >> Thanks, also +1 to supporting points 2 and 3.
> >>
> >>> On Mon, Jun 13, 2016 at 10:33 AM, <meghdoot_b@yahoo.com.invalid>
> wrote:
> >>>
> >>> Got it. Thx Max.
> >>> Renan will starting working on 2 and 3, involving and updating you all.
> >>>
> >>> Thx
> >>>
> >>> Sent from my iPhone
> >>>
> >>>>> On Jun 13, 2016, at 10:09 AM, Maxim Khutornenko <maxim@apache.org>
> >>>> wrote:
> >>>>
> >>>> David pinged me offline suggesting I misread the proposal for (2) as
> >>>> an attempt to support multiple executors per task rather than "per
> >>>> cluster". If that's the case, I am happy to change my vote for (2) to
> >>>> +1. I don't see much harm (aside from possible UI inconsistencies)
> >>>> from multiple executor types sharing the same cluster.
> >>>>
> >>>>> On Mon, Jun 13, 2016 at 9:54 AM,  <meghdoot_b@yahoo.com.invalid>
> >> wrote:
> >>>>> I will leave Renan for a detailed response.
> >>>>>
> >>>>> If aurora can schedule Job A with thermos and Job B that requires
a
> >>> different executor that is a first class concept in mesos, why is
> there a
> >>> objection? Renan's patch of replacing thermos executor is already in
> >>> aurora. However right now aurora can statically configure only one
> >>> executor. And so we want a job submission dynamically can call out the
> >>> executor type.
> >>>>> Please clarify your views.
> >>>>> Right now the docker integration story in mesos is weak (current
and
> >>> also the new universal containerizer). We see much value in using
> >> executors
> >>> like
> >>>>> https://github.com/mesos/docker-compose-executor
> >>>>> With aurora jobs.
> >>>>>
> >>>>> Points 2 and points 3 make the custom executor story stronger. (For
> >>> example we will run both thermos and compose executor depending on job
> >> type)
> >>>>>
> >>>>>
> >>>>> Point 1, I agree. That is outside of aurora and most likely will
be a
> >>> shim on top of generated go thrift bindings but a helper to easily test
> >>> custom executors.
> >>>>> Last time when we reviewed with aurora team, it came out current
> >> aurora
> >>> cli is tightly coupled with thermos objects for jobs and hence keep it
> as
> >>> is.
> >>>>>
> >>>>> Thx
> >>>>>
> >>>>>
> >>>>>
> >>>>> Sent from my iPhone
> >>>>>
> >>>>>> On Jun 13, 2016, at 9:19 AM, Maxim Khutornenko <maxim@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>> I am with Rick and David on this. The (1) and (2) feel like
building
> >> a
> >>>>>> parallel universe within Aurora without a clear justification
on the
> >>>>>> long-term benefits for the community.
> >>>>>>
> >>>>>> I am strong -1 on bringing yet another language into the Aurora
> >>>>>> ecosystem without identifying the strategic upside. As David
> >>>>>> suggested, any efforts towards building the first-class REST
API
> >>>>>> instead would go a long way and will be much appreciated by
> everyone.
> >>>>>>
> >>>>>> I also don't see a reason to support multiple executors per
task.
> >> That
> >>>>>> feels like re-creating thermos in a thermos-less world.
> >>>>>>
> >>>>>> As for (3), I'd like to see more details on the mechanics here
but
> >>>>>> generally positive towards supporting more TaskInfo features
in
> >>>>>> Aurora.
> >>>>>>
> >>>>>>
> >>>>>>> On Sun, Jun 12, 2016 at 11:46 AM,  <rick@chartbeat.com>
wrote:
> >>>>>>> I generally shy away from technical goals that are based
on a
> choice
> >>> of language. What is it about a go client that can't be done by
> extending
> >>> the existing client?
> >>>>>>>
> >>>>>>>> On Jun 12, 2016, at 2:00 PM, Renan DelValle <
> >> rdelval1@binghamton.edu>
> >>> wrote:
> >>>>>>>>
> >>>>>>>> Hi David,
> >>>>>>>>
> >>>>>>>> On Fri, Jun 10, 2016 at 12:33 AM, David McLaughlin <
> >>> david@dmclaughlin.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> On Thu, Jun 9, 2016 at 5:44 PM, meghdoot bhattacharya
<
> >>>>>>>>> meghdoot_b@yahoo.com.invalid> wrote:
> >>>>>>>>>
> >>>>>>>>>> Comments inline
> >>>>>>>>>>
> >>>>>>>>>>   From: David McLaughlin <dmclaughlin@apache.org>
> >>>>>>>>>> To: dev@aurora.apache.org
> >>>>>>>>>> Sent: Thursday, June 9, 2016 5:13 PM
> >>>>>>>>>> Subject: Re: Golang Aurora lib, multiple executor
support,
> >>> integrate
> >>>>>>>>>> mesos task related fields
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Jun 9, 2016 at 2:21 PM, Renan DelValle
<
> >>> rdelval1@binghamton.edu>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hello all,
> >>>>>>>>>>>
> >>>>>>>>>>> I'd like to (re-)introduce myself. My name's
Renan DelValle and
> >>> I've
> >>>>>>>>> had
> >>>>>>>>>>> the pleasure of being part of the Aurora
community for the last
> >>> year or
> >>>>>>>>>> so.
> >>>>>>>>>>>
> >>>>>>>>>>> Last year I worked to allow Aurora to utilize
custom executors.
> >>> With
> >>>>>>>>> the
> >>>>>>>>>>> help from Bill Farner, Kevin Sweeney, and
Joshua Cohen, that
> >>> feature
> >>>>>>>>>> became
> >>>>>>>>>>> a reality and made into Aurora late last
year. (Also got to
> show
> >>> an
> >>>>>>>>> early
> >>>>>>>>>>> beta at MesosCon!)
> >>>>>>>>>>>
> >>>>>>>>>>> For this year's summer, I have some new
goals which I invite
> >>> anyone to
> >>>>>>>>>>> provide input on.
> >>>>>>>>>>>
> >>>>>>>>>>> 1. Create a Golang library that works as
an alternative to
> >>> Aurora's
> >>>>>>>>>> Python
> >>>>>>>>>>> client and Pystachio. The initial goal of
this library is to
> >>> support
> >>>>>>>>> the
> >>>>>>>>>>> most critical job related Thrift API's.
(Admin operations can
> >>> continue
> >>>>>>>>> to
> >>>>>>>>>>> be done from the Aurora CLI). Since we support
custom
> executors,
> >>> we
> >>>>>>>>> need
> >>>>>>>>>> a
> >>>>>>>>>>> way that's not so tied to thermos (as Pystachio
is) to send
> jobs
> >>>>>>>>>>> configurations to Aurora (and by extension
the custom
> executor).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> You can easily add support for a different configuration
format
> >>> without
> >>>>>>>>>> having to rewrite the CLI in Go. You'd do that
by adding your
> own
> >>> custom
> >>>>>>>>>> noun/verbs to the client or replacing existing
commands. For
> >>> example, at
> >>>>>>>>>> Twitter we have our own version of 'aurora update
start' that
> >>> plugs into
> >>>>>>>>> an
> >>>>>>>>>> internal Deploy Orchestration Service and we
have a whole other
> >>> command
> >>>>>>>>> for
> >>>>>>>>>> federated deploys. I can show you how we do
that.
> >>>>>>>>>>
> >>>>>>>>>> The first thing I thought of though was - this
seems like a
> >> perfect
> >>>>>>>>>> starting point to get serious about a HTTP+JSON
API for Aurora.
> >>> Having
> >>>>>>>>> that
> >>>>>>>>>> would make it trivial to do a CLI in any language
you want, and
> >> the
> >>>>>>>>> Python
> >>>>>>>>>> CLI would really only be there to do Pystachio
evaluation.
> >>>>>>>>>>>> CLI integrations is not useful for a
lot of different reasons.
> >>> We need
> >>>>>>>>>> API's from other orchestration points from different
components
> >>> and hence
> >>>>>>>>>> we would package it in a go library. Uber, I
believe has taken
> >> the
> >>> same
> >>>>>>>>>> route (info from this mesoscon)
> >>>>>>>>>> We are not writing a replacement cli tool. As
noted admin
> >>> operations
> >>>>>>>>> would
> >>>>>>>>>> heavily leverage current aurora cli.
> >>>>>>>>>> I think REST work has been postponed and attempted
few times in
> >>> past. We
> >>>>>>>>>> cannot wait for that.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> This is the dev list for contributions to the project,
so I
> >> assumed
> >>> we were
> >>>>>>>>> discussing adding a Go CLI to Apache Aurora. Our
API is Thrift
> and
> >>> Thrift
> >>>>>>>>> has Go bindings so you're all set for whatever custom
tooling
> >> you're
> >>>>>>>>> building. Please use the users list for feedback
on that.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> While I'm very thankful for your input on the matter,
my original
> >>> text very
> >>>>>>>> clearly said:
> >>>>>>>> "1. Create a Golang library that works as an *alternative*
to
> >>>>>>>> Aurora's Python client and Pystachio."
> >>>>>>>>
> >>>>>>>> I believe every single item I have listed has the potential
to
> >>> become a
> >>>>>>>> useful contribution to this project.
> >>>>>>>> Therefore, I feel that it is prudent continue the discussion
here
> >>> (unless
> >>>>>>>> others feel similarly).
> >>>>>>>>
> >>>>>>>> -Renan
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 2. Create support for using multiple executors
within a single
> >>> Aurora
> >>>>>>>>>>> scheduler instance. As of right now, only
a single executor can
> >>> exist
> >>>>>>>>> for
> >>>>>>>>>>> each Aurora scheduler instance. The idea
is to allow the Aurora
> >>>>>>>>> scheduler
> >>>>>>>>>>> to support multiple executors such that
they can be used
> >>> alongside one
> >>>>>>>>>>> another, providing flexibility for running
jobs.
> >>>>>>>>>>
> >>>>>>>>>> Just for my own understanding - what problems
are you solving
> >> with
> >>> your
> >>>>>>>>>> custom executor? Sorry if you've explained this
to the lists
> >>> before.
> >>>>>>>>>>
> >>>>>>>>>> I ask because I'd really like to see Aurora
stop treating the
> >>> executor
> >>>>>>>>>> config as an opaque blob and being more opinionated.
The
> >>>>>>>>> Thermos/Scheduler
> >>>>>>>>>> abstraction really hinders what type of user
experience (UI) we
> >> can
> >>>>>>>>> build,
> >>>>>>>>>> and other frameworks do a much better job by
being more
> >>> opinionated and
> >>>>>>>>>> pulling executor data into the scheduler UI.
> >>>>>>>>>>>> We probably don't need to debate on
this point. Executor is a
> >>> first
> >>>>>>>>>> class mesos citizen and time is right for aurora
to have good
> >>> support for
> >>>>>>>>>> it.In our case, we have kubernetes like pods
modeled through
> >>>>>>>>> docker-compose
> >>>>>>>>>> and the executor manages that. Scheduler UI
main features should
> >>> not get
> >>>>>>>>>> bogged down or be held back for a particular
executor. That
> feels
> >>>>>>>>>> incredibly restrictive. If a special UI mode
for thermos
> executor
> >>> is
> >>>>>>>>>> created, that should be fine. We do have to
differentiate the
> >>> scheduler
> >>>>>>>>> and
> >>>>>>>>>> thermos and aurora team has done a great job
of not hard
> coupling
> >>> the
> >>>>>>>>> two.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 3. I'd like to add support for some first
class Mesos task
> >> related
> >>>>>>>>>> fields.
> >>>>>>>>>>> These would be optional and/or have defaults
so that the
> regular
> >>> Aurora
> >>>>>>>>>> CLI
> >>>>>>>>>>> does not break. One of the examples I'm
interested in
> >> integrating
> >>> is
> >>>>>>>>> the
> >>>>>>>>>>> Mesos Fetcher so that resources can be downloaded
into the
> >>> sandbox that
> >>>>>>>>>> the
> >>>>>>>>>>> custom executor may need. (The executor
path will never be
> >>> exposed as
> >>>>>>>>>> this
> >>>>>>>>>>> will be defined serverside and be static).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Wouldn't the mesos-fetcher call just be another
process to be
> >>> passed to
> >>>>>>>>>> your executor? I have to admit I don't know
enough about how the
> >>> Mesos
> >>>>>>>>>> fetcher works.
> >>>>>>>>>>>> Apache Mesos - Fetcher
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> We may add other first class fields.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> If anyone has any feedback on these, I'd
be very glad to hear
> >> it.
> >>>>>>>>>>>
> >>>>>>>>>>> That's it for me for now, thanks!
> >>>>>>>>>>>
> >>>>>>>>>>> -Renan
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> |
> >>>>>>>>>> |
> >>>>>>>>>> |
> >>>>>>>>>> |  |    |
> >>>>>>>>>>
> >>>>>>>>>> |
> >>>>>>>>>>
> >>>>>>>>>> |
> >>>>>>>>>> |
> >>>>>>>>>> |  |
> >>>>>>>>>> Apache Mesos - Fetcher
> >>>>>>>>>> |  |
> >>>>>>>>>>
> >>>>>>>>>> |
> >>>>>>>>>>
> >>>>>>>>>> |
> >
> >
> >
> >
> > --
> > Cheers,
> >
> > Zhitao Li
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message