Mailing-List: contact dev-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@aurora.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAOTkfX4i79m_RudqF4BkH9DPbbaFHSCsaAZKixd_q65MFTW-ng@mail.gmail.com>
References: 
 <CAFWq12XiTRfLE6zV2Z5uFAYR+y09=uDWr=SVgHz3U1V5z7_0sQ@mail.gmail.com>
	<CANMwFmUCCGb+6Ynm4_4Ca94+eNDVt6FMmbNZwFZr2jLpzLXe3Q@mail.gmail.com>
	<CAGofrHM13ayrtxso4ao5JDfR29C7tG53E3gS3HzAidNXK6QU0g@mail.gmail.com>
	<1459508323332.10017@blue-yonder.com>
	<CAJ4qn4=-qSxMyfjF2toM6zmU8yR-7ejpJmBVouqCB7=xVZf3aA@mail.gmail.com>
	<CAMnduq_wtE0aOQLP=XDd9zwz7aqT1jSnLO=c3x4nGbVQEy7i-w@mail.gmail.com>
	<CAFWq12WRLrKxUfWtsyTREXGe6hDppmdNaFdcBmHE5FpvRBW48w@mail.gmail.com>
	<CAMnduq_XgqjYzj_9w_--Kw3_r4BzPDHJha-RYD3H8ZbRR3QiLg@mail.gmail.com>
	<1459798968776.43791@blue-yonder.com>
	<CAOTkfX4i79m_RudqF4BkH9DPbbaFHSCsaAZKixd_q65MFTW-ng@mail.gmail.com>
Date: Mon, 4 Apr 2016 13:47:58 -0700
Message-ID: 
 <CAFWq12UcCfOPZA=EHOg5w3QXXJrHEybSatYcMNaaa9E6hS7Cyw@mail.gmail.com>
Subject: Re: Are we ready to remove the observer?
From: Bill Farner <wfarner@apache.org>
To: dev@aurora.apache.org
Content-Type: multipart/alternative; boundary=001a1136857203df1d052faed8bc

--001a1136857203df1d052faed8bc
Content-Type: text/plain; charset=UTF-8

We clearly have different experiences - i've never really benefited from
viewing the process graph, as most jobs have very simple sequences that
could be easily explained by a text file in the sandbox.  On the contrary,
i've encountered people confused by the process graph, the observer, and
sandbox browsing...so i must respectfully disagree that it is universally
appreciated.

What i'm trying to achieve is simplicity.  The observer is an extra moving
part, and another thing for operators to understand and maintain.  It also
couples Aurora to one relatively specific way of running tasks, which makes
it difficult to open new use cases like Docker tasks.  Removing the
observer starts to pull on a thread of complexity that i don't think Aurora
benefits much from, for example state checkpointing by the executor.

My goal is not to apply pressure, but to perform a gut check.  If the
answer is "No", that's fine.

On Mon, Apr 4, 2016 at 1:01 PM, Maxim Khutornenko <maxim@apache.org> wrote:

> I am with Josh on this one. Thermos Observer UI (and especially its
> process graph) is one of the features universally appreciated by our
> customers. I am all for deprecating the Observer but only in way that
> retains parity with the existing functionality and hopefully enhances
> it. What are we trying to achieve here that would justify losing some
> of our feature set?
>
> On Mon, Apr 4, 2016 at 12:42 PM, Erb, Stephan
> <Stephan.Erb@blue-yonder.com> wrote:
> > Have you recently looked at the Mesos UI, Joshua? It offers sandbox
> browsing similar to the chroot link of Thermos. So at least you don't have
> to do SSH into any box. We could link to that Mesos UI instead of the
> Thermos one, and Mesos could then serve a nice index.html that contains the
> content that was formerly served by Thermos.
> >
> > When dropping Thermos and relying on Mesos instead, we could profit from
> the recent addition such as authentication.
> >
> >
> > ________________________________________
> > From: Joshua Cohen <jcohen@apache.org>
> > Sent: Monday, April 4, 2016 18:42
> > To: dev@aurora.apache.org
> > Subject: Re: Are we ready to remove the observer?
> >
> > If you're suggesting just going to the task directory and pulling them
> out
> > of the executor logs. Yes, I could ssh into the host the task is running
> on
> > and grep the task directory out of the mesos agent logs and then trawl
> the
> > logs (or cat task.json), but that's much more effort than going to the
> > observer's task UI (i.e. it'd take a minute, rather than a few seconds).
> > I'd also posit that it's much easier for new Aurora operators to come to
> > grips with the process tree via the UI rather than a JSON blob.
> >
> > If you're suggesting something else (i.e. new UI to expose these separate
> > from the Observer), I'm fine with that, that's what I was implying above
> > would be necessary before I think we could retire the Observer.
> >
> > A counter question: do people feel that updating/deploying the Observer
> is
> > a major obstacle? I know we've got the process well automated, so it's
> > relatively painless. I'd love to replace the Observer with something
> > better, but I don't feel like it's a major drag on our productivity as it
> > exists today to warrant killing it off entirely. My opinion may be
> colored
> > by the deploy automation we have in place though!
> >
> > On Mon, Apr 4, 2016 at 9:32 AM, Bill Farner <wfarner@apache.org> wrote:
> >
> >> >
> >> > 2) Providing an easy view of a process's command-line
> >> > 3) Providing a holistic view of the task config
> >>
> >>
> >> Just to check my understanding - these could be trivially handled in
> >> text/log format, right?
> >>
> >> On Mon, Apr 4, 2016 at 9:30 AM, Joshua Cohen <jcohen@apache.org> wrote:
> >>
> >> > I'm -1 on this until we have an actual replacement for the Observer. I
> >> > think that the observer provides significant value outside of just
> >> sandbox
> >> > browsing:
> >> >
> >> > 1) Exporting task-level statistics.
> >> > 2) Providing an easy view of a process's command-line
> >> > 3) Providing a holistic view of the task config
> >> > 4) Real time utilization stats
> >> >
> >> > As a cluster operator, I use all of these features on a daily basis
> >> > (especially when I'm on call) in addition to sandbox browsing, so I
> don't
> >> > think that these uses cases are that rare.
> >> >
> >> > On Fri, Apr 1, 2016 at 6:55 AM, Steve Niemitz <sniemitz@apache.org>
> >> wrote:
> >> >
> >> > > The per-process stats have never been very useful to us (since they
> >> don't
> >> > > work for docker), however, even being able to see the processes that
> >> are
> >> > > running, how many times they've restarted, when they launched, etc
> is
> >> > > invaluable.
> >> > >
> >> > > I think there would be big pushback from users if they were to lose
> the
> >> > > functionality it provided currently (beyond log viewing).
> >> > >
> >> > > On Fri, Apr 1, 2016 at 6:58 AM, Erb, Stephan <
> >> > Stephan.Erb@blue-yonder.com>
> >> > > wrote:
> >> > >
> >> > > > From an operator and Aurora developer perspective, it would be
> really
> >> > > > great to get rid of the thermos observer quickly.
> >> > > >
> >> > > > However, from a user perspective the usability gap between
> observer
> >> and
> >> > > > plain Mesos sandbox browsing is quite large right now. I agree
> with
> >> > > > Benjamin here that it would probably work if we generate html
> pages
> >> > ready
> >> > > > for user consumption.
> >> > > >
> >> > > > These are the relevant tickets in our tracker:
> >> > > > * https://issues.apache.org/jira/browse/AURORA-725
> >> > > > * https://issues.apache.org/jira/browse/AURORA-777
> >> > > >
> >> > > > ________________________________________
> >> > > > From: benley@gmail.com <benley@gmail.com>
> >> > > > Sent: Friday, April 1, 2016 02:35
> >> > > > To: dev@aurora.apache.org
> >> > > > Subject: Re: Are we ready to remove the observer?
> >> > > >
> >> > > > Is there any chance we can keep the per-process cpu and ram
> >> utilization
> >> > > > stats?  That's one of the coolest things about aurora, imo.  The
> >> > executor
> >> > > > is already writing those checkpoints inside the mesos sandbox (I
> >> > think?),
> >> > > > so perhaps it could also produce the html pages that the observer
> >> > > currently
> >> > > > renders?
> >> > > >
> >> > > > On Thu, Mar 31, 2016 at 4:33 PM Zhitao Li <zhitaoli.cs@gmail.com>
> >> > wrote:
> >> > > >
> >> > > > > +1.
> >> > > > >
> >> > > > > On Thu, Mar 31, 2016 at 4:11 PM, Bill Farner <
> wfarner@apache.org>
> >> > > wrote:
> >> > > > >
> >> > > > > > Assuming that the vast majority of utility provided by the
> >> observer
> >> > > is
> >> > > > > > sandbox/log browsing - can we remove it and link to sandbox
> >> > browsing
> >> > > > that
> >> > > > > > mesos provides?
> >> > > > > >
> >> > > > > > The rest of the information could be (or already is) logged in
> >> the
> >> > > > > sandbox
> >> > > > > > for the rare debugging scenarios that call for it.
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Cheers,
> >> > > > >
> >> > > > > Zhitao Li
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>

--001a1136857203df1d052faed8bc--