aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Khutornenko <ma...@apache.org>
Subject Re: Are we ready to remove the observer?
Date Tue, 05 Apr 2016 00:10:18 GMT
Sorry, I should have tried harder explaining myself. There is no
process to host the observer UI when a task terminates. We still want
(and arguably more so) to look at terminal task details but since
there is no process left to host the http server, there is no way to
access that data in the current way of things.

On Mon, Apr 4, 2016 at 3:54 PM, Bill Farner <wfarner@apache.org> wrote:
>>
>> It falls apart for terminal tasks when executor process is not running
>> anymore.
>
>
> This sounds important...can you recall what would not work in that
> scenario?  I figured it would work ~identically because the observer
> follows the lifecycle of the task sandbox directory.
>
> On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <maxim@apache.org> wrote:
>
>> I think we've discussed this option before. It falls apart for
>> terminal tasks when executor process is not running anymore.
>>
>> One of the possible ways forward could be extending Mesos UI to
>> opportunistically consume task data periodically dumped by an executor
>> into a json file. That could cover the functionality gap created by
>> killing the observer and let other frameworks customize their task
>> views in a standard and pluggable way.
>>
>> On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sniemitz@apache.org> wrote:
>> > It seems like the easiest path forward would be to have the executor
>> itself
>> > host the observer web UI, if the HTTP port for the UI were configured as
>> > just another port on the task, the aurora UI could just link to /mname
>> for
>> > that instance.
>> >
>> > I think the overall "what is running on this machine" view the observer
>> > displays (if you go to it without a task ID) is much less useful and
>> could
>> > probably be removed without much sadness.
>> >
>> > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wfarner@apache.org> wrote:
>> >
>> >> >
>> >> > why don't we revisit the problem from the other direction and see if
>> we
>> >> > can remove checkpoints?
>> >>
>> >>
>> >> Simplicity, again :-)  If it turns out we don't need the observer
>> anyhow,
>> >> it saves a lot of time.  I'm just poking at different parts to make
>> sure we
>> >> can still justify their weight.
>> >>
>> >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zmanji@apache.org> wrote:
>> >>
>> >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfarner@apache.org>
>> wrote:
>> >> >
>> >> > > We clearly have different experiences - i've never really benefited
>> >> from
>> >> > > viewing the process graph, as most jobs have very simple sequences
>> that
>> >> > > could be easily explained by a text file in the sandbox.  On the
>> >> > contrary,
>> >> > > i've encountered people confused by the process graph, the observer,
>> >> and
>> >> > > sandbox browsing...so i must respectfully disagree that it is
>> >> universally
>> >> > > appreciated.
>> >> > >
>> >> > > What i'm trying to achieve is simplicity.  The observer is an
extra
>> >> > moving
>> >> > > part, and another thing for operators to understand and maintain.
>> It
>> >> > also
>> >> > > couples Aurora to one relatively specific way of running tasks,
>> which
>> >> > makes
>> >> > > it difficult to open new use cases like Docker tasks.  Removing
the
>> >> > > observer starts to pull on a thread of complexity that i don't
think
>> >> > Aurora
>> >> > > benefits much from, for example state checkpointing by the executor.
>> >> > >
>> >> > > My goal is not to apply pressure, but to perform a gut check.
 If
>> the
>> >> > > answer is "No", that's fine.
>> >> > >
>> >> >
>> >> >
>> >> > Bill,
>> >> >
>> >> > I think you are pulling on the right thread here but I think
>> revisiting
>> >> the
>> >> > observer is the wrong way of approaching the problem. I also agree
>> that
>> >> > Aurora doesn't benefit much from state checkpointing by the executor
>> and
>> >> > the observer is an extension of that since it provides a read only
>> human
>> >> > friendly view of the data in the checkpoints. However, instead of
>> >> removing
>> >> > the observer (and degrading the UX around accessing the data in the
>> >> > checkpoints), why don't we revisit the problem from the other
>> direction
>> >> and
>> >> > see if we can remove checkpoints?
>> >> >
>> >> >
>> >> > --
>> >> > Zameer Manji
>> >> >
>> >>
>>

Mime
View raw message