aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Farner <wfar...@apache.org>
Subject Re: Are we ready to remove the observer?
Date Mon, 04 Apr 2016 22:54:09 GMT
>
> It falls apart for terminal tasks when executor process is not running
> anymore.


This sounds important...can you recall what would not work in that
scenario?  I figured it would work ~identically because the observer
follows the lifecycle of the task sandbox directory.

On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <maxim@apache.org> wrote:

> I think we've discussed this option before. It falls apart for
> terminal tasks when executor process is not running anymore.
>
> One of the possible ways forward could be extending Mesos UI to
> opportunistically consume task data periodically dumped by an executor
> into a json file. That could cover the functionality gap created by
> killing the observer and let other frameworks customize their task
> views in a standard and pluggable way.
>
> On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sniemitz@apache.org> wrote:
> > It seems like the easiest path forward would be to have the executor
> itself
> > host the observer web UI, if the HTTP port for the UI were configured as
> > just another port on the task, the aurora UI could just link to /mname
> for
> > that instance.
> >
> > I think the overall "what is running on this machine" view the observer
> > displays (if you go to it without a task ID) is much less useful and
> could
> > probably be removed without much sadness.
> >
> > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wfarner@apache.org> wrote:
> >
> >> >
> >> > why don't we revisit the problem from the other direction and see if
> we
> >> > can remove checkpoints?
> >>
> >>
> >> Simplicity, again :-)  If it turns out we don't need the observer
> anyhow,
> >> it saves a lot of time.  I'm just poking at different parts to make
> sure we
> >> can still justify their weight.
> >>
> >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zmanji@apache.org> wrote:
> >>
> >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfarner@apache.org>
> wrote:
> >> >
> >> > > We clearly have different experiences - i've never really benefited
> >> from
> >> > > viewing the process graph, as most jobs have very simple sequences
> that
> >> > > could be easily explained by a text file in the sandbox.  On the
> >> > contrary,
> >> > > i've encountered people confused by the process graph, the observer,
> >> and
> >> > > sandbox browsing...so i must respectfully disagree that it is
> >> universally
> >> > > appreciated.
> >> > >
> >> > > What i'm trying to achieve is simplicity.  The observer is an extra
> >> > moving
> >> > > part, and another thing for operators to understand and maintain.
> It
> >> > also
> >> > > couples Aurora to one relatively specific way of running tasks,
> which
> >> > makes
> >> > > it difficult to open new use cases like Docker tasks.  Removing the
> >> > > observer starts to pull on a thread of complexity that i don't think
> >> > Aurora
> >> > > benefits much from, for example state checkpointing by the executor.
> >> > >
> >> > > My goal is not to apply pressure, but to perform a gut check.  If
> the
> >> > > answer is "No", that's fine.
> >> > >
> >> >
> >> >
> >> > Bill,
> >> >
> >> > I think you are pulling on the right thread here but I think
> revisiting
> >> the
> >> > observer is the wrong way of approaching the problem. I also agree
> that
> >> > Aurora doesn't benefit much from state checkpointing by the executor
> and
> >> > the observer is an extension of that since it provides a read only
> human
> >> > friendly view of the data in the checkpoints. However, instead of
> >> removing
> >> > the observer (and degrading the UX around accessing the data in the
> >> > checkpoints), why don't we revisit the problem from the other
> direction
> >> and
> >> > see if we can remove checkpoints?
> >> >
> >> >
> >> > --
> >> > Zameer Manji
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message