aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Farner <wfar...@apache.org>
Subject Re: Are we ready to remove the observer?
Date Tue, 05 Apr 2016 00:39:00 GMT
>
> There is no process to host the observer UI when a task terminates.


Aha, i didn't realize that was in reference to Steve's comment "have the
executor itself host the observer web UI".
I agree, that approach is encumbered for terminal tasks.

On Mon, Apr 4, 2016 at 5:10 PM, Maxim Khutornenko <maxim@apache.org> wrote:

> Sorry, I should have tried harder explaining myself. There is no
> process to host the observer UI when a task terminates. We still want
> (and arguably more so) to look at terminal task details but since
> there is no process left to host the http server, there is no way to
> access that data in the current way of things.
>
> On Mon, Apr 4, 2016 at 3:54 PM, Bill Farner <wfarner@apache.org> wrote:
> >>
> >> It falls apart for terminal tasks when executor process is not running
> >> anymore.
> >
> >
> > This sounds important...can you recall what would not work in that
> > scenario?  I figured it would work ~identically because the observer
> > follows the lifecycle of the task sandbox directory.
> >
> > On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <maxim@apache.org>
> wrote:
> >
> >> I think we've discussed this option before. It falls apart for
> >> terminal tasks when executor process is not running anymore.
> >>
> >> One of the possible ways forward could be extending Mesos UI to
> >> opportunistically consume task data periodically dumped by an executor
> >> into a json file. That could cover the functionality gap created by
> >> killing the observer and let other frameworks customize their task
> >> views in a standard and pluggable way.
> >>
> >> On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sniemitz@apache.org>
> wrote:
> >> > It seems like the easiest path forward would be to have the executor
> >> itself
> >> > host the observer web UI, if the HTTP port for the UI were configured
> as
> >> > just another port on the task, the aurora UI could just link to /mname
> >> for
> >> > that instance.
> >> >
> >> > I think the overall "what is running on this machine" view the
> observer
> >> > displays (if you go to it without a task ID) is much less useful and
> >> could
> >> > probably be removed without much sadness.
> >> >
> >> > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wfarner@apache.org>
> wrote:
> >> >
> >> >> >
> >> >> > why don't we revisit the problem from the other direction and
see
> if
> >> we
> >> >> > can remove checkpoints?
> >> >>
> >> >>
> >> >> Simplicity, again :-)  If it turns out we don't need the observer
> >> anyhow,
> >> >> it saves a lot of time.  I'm just poking at different parts to make
> >> sure we
> >> >> can still justify their weight.
> >> >>
> >> >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zmanji@apache.org>
> wrote:
> >> >>
> >> >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfarner@apache.org>
> >> wrote:
> >> >> >
> >> >> > > We clearly have different experiences - i've never really
> benefited
> >> >> from
> >> >> > > viewing the process graph, as most jobs have very simple
> sequences
> >> that
> >> >> > > could be easily explained by a text file in the sandbox.
 On the
> >> >> > contrary,
> >> >> > > i've encountered people confused by the process graph, the
> observer,
> >> >> and
> >> >> > > sandbox browsing...so i must respectfully disagree that it
is
> >> >> universally
> >> >> > > appreciated.
> >> >> > >
> >> >> > > What i'm trying to achieve is simplicity.  The observer is
an
> extra
> >> >> > moving
> >> >> > > part, and another thing for operators to understand and maintain.
> >> It
> >> >> > also
> >> >> > > couples Aurora to one relatively specific way of running
tasks,
> >> which
> >> >> > makes
> >> >> > > it difficult to open new use cases like Docker tasks.  Removing
> the
> >> >> > > observer starts to pull on a thread of complexity that i
don't
> think
> >> >> > Aurora
> >> >> > > benefits much from, for example state checkpointing by the
> executor.
> >> >> > >
> >> >> > > My goal is not to apply pressure, but to perform a gut check.
 If
> >> the
> >> >> > > answer is "No", that's fine.
> >> >> > >
> >> >> >
> >> >> >
> >> >> > Bill,
> >> >> >
> >> >> > I think you are pulling on the right thread here but I think
> >> revisiting
> >> >> the
> >> >> > observer is the wrong way of approaching the problem. I also agree
> >> that
> >> >> > Aurora doesn't benefit much from state checkpointing by the
> executor
> >> and
> >> >> > the observer is an extension of that since it provides a read
only
> >> human
> >> >> > friendly view of the data in the checkpoints. However, instead
of
> >> >> removing
> >> >> > the observer (and degrading the UX around accessing the data in
the
> >> >> > checkpoints), why don't we revisit the problem from the other
> >> direction
> >> >> and
> >> >> > see if we can remove checkpoints?
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Zameer Manji
> >> >> >
> >> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message