aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Farner <wfar...@apache.org>
Subject Re: Are we ready to remove the observer?
Date Mon, 04 Apr 2016 21:23:34 GMT
>
> why don't we revisit the problem from the other direction and see if we
> can remove checkpoints?


Simplicity, again :-)  If it turns out we don't need the observer anyhow,
it saves a lot of time.  I'm just poking at different parts to make sure we
can still justify their weight.

On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zmanji@apache.org> wrote:

> On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfarner@apache.org> wrote:
>
> > We clearly have different experiences - i've never really benefited from
> > viewing the process graph, as most jobs have very simple sequences that
> > could be easily explained by a text file in the sandbox.  On the
> contrary,
> > i've encountered people confused by the process graph, the observer, and
> > sandbox browsing...so i must respectfully disagree that it is universally
> > appreciated.
> >
> > What i'm trying to achieve is simplicity.  The observer is an extra
> moving
> > part, and another thing for operators to understand and maintain.  It
> also
> > couples Aurora to one relatively specific way of running tasks, which
> makes
> > it difficult to open new use cases like Docker tasks.  Removing the
> > observer starts to pull on a thread of complexity that i don't think
> Aurora
> > benefits much from, for example state checkpointing by the executor.
> >
> > My goal is not to apply pressure, but to perform a gut check.  If the
> > answer is "No", that's fine.
> >
>
>
> Bill,
>
> I think you are pulling on the right thread here but I think revisiting the
> observer is the wrong way of approaching the problem. I also agree that
> Aurora doesn't benefit much from state checkpointing by the executor and
> the observer is an extension of that since it provides a read only human
> friendly view of the data in the checkpoints. However, instead of removing
> the observer (and degrading the UX around accessing the data in the
> checkpoints), why don't we revisit the problem from the other direction and
> see if we can remove checkpoints?
>
>
> --
> Zameer Manji
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message