aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Erb <s...@apache.org>
Subject Re: Redesign of the Aurora UI
Date Sat, 23 Sep 2017 05:39:27 GMT
+1 for switching to react. I believe this will reduce maintenance
effort for us in the long term. 

On Fri, 2017-09-22 at 15:19 -0700, David McLaughlin wrote:
> Just a heads up: Facebook have decided to relicense React and Jest as
> MIT:
> 
> https://code.facebook.com/posts/300798627056246
> 
> So I'll be adopting React and dropping the Preact-compat stuff asap.
> The
> effect to our application code is minimal (if any), but tests will
> need to
> be rewritten. So I'll probably hold off on further patches until the
> relicense is complete.
> 
> On Fri, Aug 18, 2017 at 8:13 PM, David McLaughlin <david@dmclaughlin.
> com>
> wrote:
> 
> > Good to know they made the decision. My plan is to move forward
> > with
> > Preact, with the major question how to do unit testing since we
> > can't use
> > jest either. The first patch should land early next week.
> > 
> > On Fri, Aug 18, 2017 at 6:29 PM, Joshua Cohen <jcohen@apache.org>
> > wrote:
> > 
> > > FB came back w/ their response to relicense React today:
> > > https://code.facebook.com/posts/112130496157735/explaining-
> > > react-s-license
> > > 
> > > tl;dr: they're not relicensing React. So... if we do go down this
> > > path,
> > > we'll have to investigate alternatives to React (Preact, Vue,
> > > etc.) or
> > > potentially release this as a separate project outside of the ASF
> > > that can
> > > serve as a drop in UI replacement.
> > > 
> > > On Sun, Jul 23, 2017 at 4:48 PM, Erb, Stephan <
> > > Stephan.Erb@blue-yonder.com>
> > > wrote:
> > > 
> > > > A big +1 from me as well. We have not touched or updated the
> > > > existing UI
> > > > for quite some time, which is a bad sign for code health.
> > > > 
> > > > I would even be OK with a couple of bigger initial code dumps.
> > > > I am not
> > > > really a web-developer, so a working piece of code to play
> > > > around with
> > > > would probably be the fastest way to get up to speed with the
> > > > tech and
> > > 
> > > its
> > > > usage in Aurora.
> > > > 
> > > > Thanks a lot for driving this, David!
> > > > 
> > > > On 21.07.17, 07:00, "Kai Huang" <texasred2013@hotmail.com>
> > > > wrote:
> > > > 
> > > >     David - Sure, let's sync on the work when you are ready.
> > > > 
> > > > 
> > > >     ________________________________
> > > >     From: David McLaughlin <dmclaughlin@apache.org>
> > > >     Sent: Wednesday, July 19, 2017 14:10
> > > >     To: dev@aurora.apache.org
> > > >     Subject: Re: Redesign of the Aurora UI
> > > > 
> > > >     Thanks for the feedback!
> > > > 
> > > >     Joshua - I haven't tried to drop in Preact yet, but I was
> > > > also
> > > > planning to
> > > >     throw away the prototype and starting again when it came to
> > > > upstreaming it,
> > > >     so as part of that we can just address incompatibilities as
> > > > we go.
> > > 
> > > If
> > > > I was
> > > >     to guess, then the only significant impact on my prototype
> > > > would
> > > > probably
> > > >     be the reactable plugin I was using (replacement for
> > > > Angular's
> > > >     smart-table). But longer term I do have concerns about
> > > > moving away
> > > 
> > > from
> > > >     what is a constantly improving and healthy ecosystem around
> > > > React.
> > > 
> > > So
> > > > most
> > > >     likely I'll hold off until a decision is made there one way
> > > > or the
> > > > other
> > > >     (which should be within a week).
> > > > 
> > > > 
> > > >     Kai - I'd be happy to coordinate and collaborate on this
> > > > with
> > > 
> > > others.
> > > > Let
> > > >     me try and finish up the CSS/UX of the pages in my
> > > > prototype and
> > > 
> > > from
> > > > there
> > > >     we can sync on who does what. Does that sound good?
> > > > 
> > > > 
> > > > 
> > > > 
> > > >     On Wed, Jul 19, 2017 at 1:56 PM, Kai Huang <
> > > 
> > > texasred2013@hotmail.com>
> > > > wrote:
> > > > 
> > > >     > Just a few thoughts as an aurora developer and operator:
> > > >     >
> > > >     >
> > > >     > From my experiences with Aurora users, some persisting
> > > > complaints
> > > > are:
> > > >     >
> > > >     >   1.  The current UI is not very intuitive for the users
> > > > to
> > > > understand the
> > > >     > task lifecycle, resource utilization of their job.
> > > >     >   2.  Often times users are unaware of the new
> > > > features/changes in
> > > > Aurora
> > > >     > Scheduler/Executor, which leads to a lot of misuse of the
> > > > system.
> > > >     >   3.  Users have preferences on the appearance of the
> > > > scheduler/thermos UI
> > > >     > due to special use cases, and ask us to customize for
> > > > them(or
> > > 
> > > start
> > > > to
> > > >     > write their own UI, which is often not recommended).
> > > >     >
> > > >     >
> > > >     > The other major issue I see in the current UI is that
> > > > it's built
> > > 
> > > on
> > > > an
> > > >     > obsolete tech stack(AngularJS) that has all the binaries
> > > > and
> > > > dependencies
> > > >     > in the repo. From a developer's perspective, it's a big
> > > > burden to
> > > >     > maintain/test the code, and make fast iterations on it.
> > > >     >
> > > >     >
> > > >     > Currently the scheduler UI is readonly, and mainly
> > > > designed for
> > > > debugging
> > > >     > purposes. We could have done much better to make the UI
> > > > more
> > > > friendly to
> > > >     > the end user, empower them to discover, understand and
> > > > use all the
> > > > Aurora
> > > >     > features, and give them more insights into the their
> > > > jobs, or even
> > > > the
> > > >     > entire cluster. I love the idea that redesign the UI with
> > > > a modern
> > > > stack,
> > > >     > and more importantly, every single part of the
> > > > application is a
> > > > module that
> > > >     > you could take your own customization.
> > > >     >
> > > >     >
> > > >     > As for the development strategy, I'm in favor of the
> > > > incremental
> > > > approach
> > > >     > that posts one page at a time. The main benefit is that
> > > > we are
> > > > educating
> > > >     > the developers while iterating on it, and this will
> > > > improve the
> > > > adoption
> > > >     > rate in the long term.
> > > >     >
> > > >     >
> > > >     > Overall I'm very interested in this work, and would like
> > > > to
> > > > collaborate
> > > >     > with David to redesign and improve the UI.
> > > >     >
> > > >     >
> > > >     > ________________________________
> > > >     > From: Joshua Cohen <jcohen@apache.org>
> > > >     > Sent: Wednesday, July 19, 2017 11:39
> > > >     > To: dev@aurora.apache.org
> > > >     > Subject: Re: Redesign of the Aurora UI
> > > >     >
> > > >     > I think this looks great overall! I'm super excited to
> > > > see the UI
> > > > get some
> > > >     > love (and to set us up for better iteration on the UI
> > > > going
> > > > forward). My
> > > >     > biggest concern, of course, is the current hubbub vis-a-
> > > > vis Apache
> > > > and the
> > > >     > BSD-3+Patents license[1]. Have you tried running this
> > > > against
> > > 
> > > Preact
> > > > to
> > > >     > confirm compatibility/performance? Also note that other
> > > > Facebook
> > > > libraries
> > > >     > have the same license problem (e.g. Immutable, Jest), so
> > > > unless FB
> > > > changes
> > > >     > their patent grant clause, I imagine we'd have to find
> > > 
> > > alternatives
> > > > to
> > > >     > those as well. If only we had landed this a week ago, we
> > > > could've
> > > > been
> > > >     > grandfathered in on the license front :(.
> > > >     >
> > > >     > As far as the options for landing this, I'll leave that
> > > > up to more
> > > > active
> > > >     > reviewers, but my gut says that smaller reviews will make
> > > > this
> > > > easier to
> > > >     > parse, especially for those unfamiliar with React. That
> > > > said,
> > > > perhaps we
> > > >     > could go with an alternate method for reviewing here,
> > > > where people
> > > > review
> > > >     > against your fork directly and only when they're
> > > > comfortable do
> > > 
> > > you
> > > > post
> > > >     > the whole patch to reviewboard for what should, by that
> > > > point, be
> > > 
> > > a
> > > > rubber
> > > >     > stamp review?
> > > >     >
> > > >     > In general, fantastic work, David!
> > > >     >
> > > >     > [1] https://issues.apache.org/jira/browse/LEGAL-319
> > > >     >
> > > >     > On Wed, Jul 19, 2017 at 12:55 AM, David McLaughlin <
> > > > dmclaughlin@apache.org
> > > >     > >
> > > >     > wrote:
> > > >     >
> > > >     > > Hey all,
> > > >     > >
> > > >     > > At Twitter we have had a long-standing desire to be
> > > > able to put
> > > > custom
> > > >     > > widgets and other UX enhancements into the Aurora UI.
> > > > Recent
> > > > prototype
> > > >     > work
> > > >     > > to do this in a clean way has proved fruitful and I'd
> > > > like to
> > > > present
> > > >     > this
> > > >     > > approach to the community and get feedback on the
> > > > overall
> > > 
> > > approach.
> > > >     > >
> > > >     > > The basic approach is simple:
> > > >     > >
> > > >     > > 1) Use the node plugin for gradle to bootstrap a modern
> > > > web
> > > > development
> > > >     > > build system using webpack and npm.
> > > >     > > 2) Use a modern JS view library like ReactJS (or Preact
> > > 
> > > depending
> > > > on the
> > > >     > > Facebook+Patents license issues for Apache projects) to
> > > > have
> > > 
> > > all UI
> > > >     > > components defined as ES6 modules.
> > > >     > > 3) Use webpack to set up a custom plugin directory that
> > > > modules
> > > > should be
> > > >     > > loaded from first. This would allow anyone with custom
> > > > Scheduler
> > > > builds
> > > >     > to
> > > >     > > also drop in custom JS files to replace selected OSS UI
> > > 
> > > components.
> > > >     > > Example, you could drop a "Navigation.js" into the
> > > > plugin
> > > > directory, and
> > > >     > > the build system will use that module instead whenever
> > > > it sees a
> > > >     > > <Navigation /> used by parent components. Basically:
> > > > it's a
> > > 
> > > hybrid
> > > >     > > plugin/dependency injection (similar to Guice) approach
> > > > to
> > > > customization
> > > >     > of
> > > >     > > the UI.
> > > >     > >
> > > >     > > The nice thing here is that since every single part of
> > > > the
> > > > application
> > > >     > is a
> > > >     > > module you could take your customizations to any level
> > > > of
> > > > precision that
> > > >     > > you need. You could replace (or add) key pages or
> > > > components, or
> > > > just
> > > >     > keep
> > > >     > > it as simple as adding some helper text to one of the
> > > > pages.
> > > >     > >
> > > >     > > Some use cases this could enable:
> > > >     > >
> > > >     > > * A <ConfigurationViewer /> widget that you can replace
> > > > with
> > > >     > configuration
> > > >     > > parsing that understands special processes and/or
> > > > custom
> > > 
> > > executors.
> > > >     > > * Integration with your telemetry stacks to add
> > > > resource
> > > > utilization
> > > >     > > feedback or other performance indicators into the job
> > > > page.
> > > >     > > * Adding custom stats tracking widgets for internal
> > > > analytics.
> > > >     > >
> > > >     > >
> > > >     > > Some points from my prototype work:
> > > >     > >
> > > >     > > 1) Switching to npm for JS asset management allows us
> > > > to delete
> > > > the vast
> > > >     > > majority of our 3rdparty assets that we had to copy
> > > > into the
> > > 
> > > repo.
> > > > We can
> > > >     > > most likely extend this to remove all of them,
> > > > including our CSS
> > > >     > frameworks
> > > >     > > like Bootstrap.
> > > >     > > 2) ReactJS was proposed before (by Joshua Cohen) and
> > > > ironically
> > > 
> > > I
> > > > was one
> > > >     > > of the main voices against. The argument I used at the
> > > > time was
> > > > lack of
> > > >     > > familiarity with the stack within the community, as
> > > > well as
> > > > concerns
> > > >     > about
> > > >     > > the complexity. So what changed? Almost the entire
> > > > Aurora team
> > > 
> > > at
> > > > Twitter
> > > >     > > has ReactJS experience now - from working on our
> > > > internal
> > > > Continuous
> > > >     > > Delivery system. I also believe that as a view layer,
> > > > ReactJS is
> > > >     > > conceptually very simple and that most of the
> > > > complexity in
> > > 
> > > these
> > > > modern
> > > >     > JS
> > > >     > > applications is in a state layer like Redux. The Aurora
> > > > UI is
> > > > currently
> > > >     > > read-only and relatively simple, and we can avoid much
> > > > of the
> > > > complexity
> > > >     > by
> > > >     > > just using component state instead.
> > > >     > > 3) Unit tests are trivial to add as part of this work.
> > > >     > > 4) I'd like to move to modern, maintainable CSS tooling
> > > > - SASS -
> > > > as part
> > > >     > of
> > > >     > > this.
> > > >     > >
> > > >     > >
> > > >     > > My prototype is (hastily) published here:
> > > >     > >
> > > >     > > https://github.com/DavidMcLaughlin/aurora/tree/js-build
> > > >     [https://avatars0.githubusercontent.com/u/56563?v=4&s=400]<
> > > > ;
> > > > https://github.com/DavidMcLaughlin/aurora/tree/js-build>;
> > > > 
> > > >     DavidMcLaughlin/aurora<https://github.com/DavidMcLaughlin/
> > > > aurora/tree/js-build>
> > > >     github.com
> > > >     aurora - Mirror of Apache Aurora
> > > > 
> > > > 
> > > > 
> > > >     > [https://avatars0.githubusercontent.com/u/56563?v=4&s=400
> > > > ]<;
> > > >     > https://github.com/DavidMcLaughlin/aurora/tree/js-build>;
> > > >     [https://avatars0.githubusercontent.com/u/56563?v=4&s=400]<
> > > > ;
> > > > https://github.com/DavidMcLaughlin/aurora/tree/js-build>;
> > > > 
> > > >     DavidMcLaughlin/aurora<https://github.com/DavidMcLaughlin/
> > > > aurora/tree/js-build>
> > > >     github.com
> > > >     aurora - Mirror of Apache Aurora
> > > > 
> > > > 
> > > > 
> > > >     >
> > > >     > DavidMcLaughlin/aurora<https://github.com/DavidMcLaughlin
> > > > /
> > > >     > aurora/tree/js-build>
> > > >     > github.com
> > > >     > aurora - Mirror of Apache Aurora
> > > >     >
> > > >     >
> > > >     >
> > > >     > >
> > > >     > >
> > > >     > > Some highlighted changes:
> > > >     > >
> > > >     > > 1) Bootstrapping the modern UI toolchain from gradle:
> > > >     > > https://github.com/DavidMcLaughlin/aurora/blob/
> > > >     > js-build/build.gradle#L452-
> > > >     > [https://avatars0.githubusercontent.com/u/56563?v=4&s=400
> > > > ]<;
> > > >     > https://github.com/DavidMcLaughlin/aurora/blob/
> > > > js-build/build.gradle#L452-
> > > >     > >
> > > >     >
> > > >     > DavidMcLaughlin/aurora<https://github.com/DavidMcLaughlin
> > > > /
> > > >     > aurora/blob/js-build/build.gradle#L452->
> > > >     > github.com
> > > >     > aurora - Mirror of Apache Aurora
> > > >     >
> > > >     >
> > > >     >
> > > >     > > L459
> > > >     > >
> > > >     > > 2) The webpack configuration to enable loading from a
> > > > plugin
> > > > directory
> > > >     > (and
> > > >     > > SASS):
> > > >     > > https://github.com/DavidMcLaughlin/aurora/blob/
> > > >     > js-build/webpack.config.js
> > > >     > [https://avatars0.githubusercontent.com/u/56563?v=4&s=400
> > > > ]<;
> > > >     > https://github.com/DavidMcLaughlin/aurora/blob/
> > > > js-build/webpack.config.js>
> > > >     >
> > > >     > DavidMcLaughlin/aurora<https://github.com/DavidMcLaughlin
> > > > /
> > > >     > aurora/blob/js-build/webpack.config.js>
> > > >     > github.com
> > > >     > aurora - Mirror of Apache Aurora
> > > >     >
> > > >     >
> > > >     >
> > > >     > >
> > > >     > > 3) The Hello, World of the plugin mechanism:
> > > >     > >
> > > >     > > https://github.com/DavidMcLaughlin/aurora/blob/
> > > > js-build/src/main/js/
> > > >     > [https://avatars0.githubusercontent.com/u/56563?v=4&s=400
> > > > ]<;
> > > >     > https://github.com/DavidMcLaughlin/aurora/blob/js-build/s
> > > > rc/
> > > 
> > > main/js/
> > > > > 
> > > > 
> > > >     >
> > > >     > DavidMcLaughlin/aurora<https://github.com/DavidMcLaughlin
> > > > /
> > > >     > aurora/blob/js-build/src/main/js/>
> > > >     > github.com
> > > >     > aurora - Mirror of Apache Aurora
> > > >     >
> > > >     >
> > > >     >
> > > >     > > components/Hello.js
> > > >     > >
> > > >     > > Is replaced by:
> > > >     > >
> > > >     > > https://github.com/DavidMcLaughlin/aurora/blob/
> > > >     > > js-build/plugin/js/components/Hello.js
> > > >     > >
> > > >     > > 4) Unit test example:
> > > >     > > https://github.com/DavidMcLaughlin/aurora/blob/
> > > > js-build/src/main/js/
> > > >     > [https://avatars0.githubusercontent.com/u/56563?v=4&s=400
> > > > ]<;
> > > >     > https://github.com/DavidMcLaughlin/aurora/blob/js-build/s
> > > > rc/
> > > 
> > > main/js/
> > > > > 
> > > > 
> > > >     >
> > > >     > DavidMcLaughlin/aurora<https://github.com/DavidMcLaughlin
> > > > /
> > > >     > aurora/blob/js-build/src/main/js/>
> > > >     > github.com
> > > >     > aurora - Mirror of Apache Aurora
> > > >     >
> > > >     >
> > > >     >
> > > >     > > components/pages/__tests__/HomePage-test.js
> > > >     > >
> > > >     > >
> > > >     > > My goal now is to take the prototype to completion and
> > > > upstream
> > > 
> > > to
> > > > Aurora
> > > >     > > OSS. If the community is in favor, we should also
> > > > discuss how to
> > > > handle
> > > >     > the
> > > >     > > rollout.
> > > >     > >
> > > >     > > I would propose two options:
> > > >     > >
> > > >     > > * a massive code-dump with the entire UI rewritten in
> > > > ReactJS.
> > > >     > > * an incremental approach where I (for example) post
> > > > one page
> > > 
> > > at a
> > > > time,
> > > >     > > allowing those unfamiliar with JS to see the step by
> > > > step
> > > 
> > > process
> > > > that
> > > >     > can
> > > >     > > demystify the tooling. The new UI would not be enabled
> > > > incrementally due
> > > >     > to
> > > >     > > the issues with "greedy" one-page frameworks.. instead
> > > > we'd just
> > > > enable
> > > >     > it
> > > >     > > at the very end. The major downside here is just
> > > > maintaining all
> > > > the
> > > >     > weight
> > > >     > > of two UI frameworks, etc. in that period of
> > > > development.
> > > >     > >
> > > >     > > Please let me know any questions, concerns or feedback.
> > > >     > >
> > > >     > >
> > > >     > > Cheers,
> > > >     > > David
> > > >     > >
> > > >     >
> > > > 
> > > > 
> > > > 
> > 
> > 

Mime
View raw message