airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Feng <fengta...@gmail.com>
Subject Re: RBAC Update
Date Thu, 29 Mar 2018 05:02:06 GMT
Great work Joy. This is awesome! I am interested in helping out the per dag
level access.  Just created a ticket to check(AIRFLOW-2267). Let me know if
you have any suggestions. I will share my proposal once I am ready.

On Fri, Mar 23, 2018 at 6:45 PM, Joy Gao <joyg@wepay.com> wrote:

> Hey guys!
>
> The RBAC UI <https://github.com/apache/incubator-airflow/pull/3015> has
> been merged to master. I'm looking forward to early adopters' feedback and
> bug reports. I also hope to have more folks helping out with the RBAC UI,
> especially with introducing DAG-Level access control, which is a feature
> that a lot of people have been asking. If you are interested in helping out
> with this effort, let's talk more!
>
> This commit will be in the 1.10.0 release, and we are going to maintain
> both UIs simultaneously for a short period of time. Once RBAC UI is stable
> and battle-tested, we will deprecate the old UI and eventually remove it
> from the repo (around Airflow 2.0.0 or 2.1.0 release). This is to prevent
> two UIs from forking into separate paths, as that would become very
> difficult to maintain.
>
> Going forward while both UIs are up, if you are making a change to any
> files in airflow/www/ (old UI), where applicable, please also make the
> change to the airflow/www_rbac/ (new UI). If you rather not make changes in
> both UIs, it is recommended that you only make the changes to the RBAC UI,
> since that is the one we are maintaining in the long term.
>
> I'm excited that the RBAC UI will be able to bring additional security to
> Airflow, and with FAB framework in place we can look into leveraging it for
> a unified set of APIs used by both UI and CLI.
>
> Joy
>
>
>
> On Thu, Feb 8, 2018 at 11:31 AM, Joy Gao <joyg@wepay.com> wrote:
>
> > Hi folks,
> >
> > I have a PR <https://github.com/apache/incubator-airflow/pull/3015> out
> > for the new UI. I've included instructions on how to test it out in the
> PR
> > description. Looking forward to your feedbacks.
> >
> > Cheers,
> > Joy
> >
> > On Fri, Dec 1, 2017 at 6:18 PM, Joy Gao <joyg@wepay.com> wrote:
> >
> >> Thanks for the background info. Would be really awesome for you to have
> >> PyPi access :D I'll make the change to have Airflow Webserver's FAB
> >> dependency pointing to my fork for the mean time.
> >>
> >> For folks who are interested in RBAC, I will be giving a talk/demo at
> the Airflow
> >> Meet-Up
> >> <https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-
> Meetup/events/244525050/>
> >> next Monday. Happy to chat afterwards about it as well :)
> >>
> >> On Thu, Nov 30, 2017 at 8:36 AM, Maxime Beauchemin <
> >> maximebeauchemin@gmail.com> wrote:
> >>
> >>> A bit of related history here:
> >>> https://github.com/dpgaspar/Flask-AppBuilder/issues/399
> >>>
> >>> On Thu, Nov 30, 2017 at 8:33 AM, Maxime Beauchemin <
> >>> maximebeauchemin@gmail.com> wrote:
> >>>
> >>> > Given I have merge rights on FAB I could probably do another round
of
> >>> > review and get your PRs through. I would really like to get the main
> >>> > maintainer's input on things that touch the core (composite-key
> >>> support) as
> >>> > he might have concerns/intuitions that we can't know about.
> >>> >
> >>> > I do not have Pypi access though so I cannot push new releases out.
I
> >>> > could ask for that.
> >>> >
> >>> > I've threatened to fork the project before, that's always an option.
> >>> I've
> >>> > noticed his involvement is sporadic and comes in bursts.
> >>> >
> >>> > In the meantime, you can have the dependency in Airflow Webserver
> >>> pointing
> >>> > straight to your fork.
> >>> >
> >>> > Max
> >>> >
> >>> > On Wed, Nov 29, 2017 at 7:02 PM, Joy Gao <joyg@wepay.com> wrote:
> >>> >
> >>> >> I just created a new webserver instance if you haven't gotten a
> >>> chance to
> >>> >> fiddle around with the new web UI and the RBAC configurations
> (thanks
> >>> >> Maxime for getting started with this earlier!):
> >>> >>
> >>> >> http://104.209.38.171:8080/
> >>> >>
> >>> >> Admin Account
> >>> >> username: admin
> >>> >> password: admin
> >>> >>
> >>> >> Read-Only Account
> >>> >> username: viewer
> >>> >> password: password
> >>> >>
> >>> >>
> >>> >> On Wed, Nov 29, 2017 at 2:58 PM, Joy Gao <joyg@wepay.com>
wrote:
> >>> >>
> >>> >> > Hi folks,
> >>> >> >
> >>> >> > Thanks for all the feedback regarding to the new Airflow Webserver
> >>> UI
> >>> >> > <https://github.com/wepay/airflow-webserver/>! I've
been actively
> >>> >> > addressing all the bugs that were raised on Github. So I want
to
> >>> take
> >>> >> this
> >>> >> > opportunity to discuss two issues coming up:
> >>> >> >
> >>> >> > The first issue is unaddressed PRs in FAB. If these PRs continue
> to
> >>> stay
> >>> >> > unaddressed, RBAC is blocked from making further progress.
If this
> >>> >> continue
> >>> >> > to be an issue, I'm inclined to fork FAB, even though it's
not
> >>> >> idealistic.
> >>> >> >
> >>> >> >
> >>> >> >    - PR/631 <https://github.com/dpgaspar/F
> lask-AppBuilder/pull/631>
> >>> >> Binary
> >>> >> >    column support (merged, unreleased)
> >>> >> >    <https://github.com/dpgaspar/Flask-AppBuilder/pull/631>
> >>> >> >    - PR/639 <https://github.com/dpgaspar/F
> lask-AppBuilder/pull/639>
> >>> >> Composite
> >>> >> >    primary key support (unmerged)
> >>> >> >    - PR/655 <https://github.com/dpgaspar/F
> lask-AppBuilder/pull/655>
> >>> >> Form
> >>> >> >    prefill support (unmerged)
> >>> >> >
> >>> >> >
> >>> >> > The second issue is an open question about the next step of
> Airflow
> >>> >> > Webserver itself. Here are the 3 potential directions we could
> >>> take, and
> >>> >> > I've added my thought on each.
> >>> >> >
> >>> >> > 1. Permanently keep Airflow Webserver as a separated package
from
> >>> >> Airflow,
> >>> >> > and treat it as another UI option. Keep `www` in Airflow.
Allow
> >>> >> development
> >>> >> > on both UIs.
> >>> >> > *I'm not a fan of this. When there is an existing UI in Airflow,
> >>> most
> >>> >> > contributors would prefer to maintain the official version
that is
> >>> >> > installed out-of-the-box. **Having a second UI outside of
Airflow
> >>> will
> >>> >> > make maintaining it very difficult, leading to an eventual
death
> of
> >>> the
> >>> >> new
> >>> >> > UI :(*
> >>> >> >
> >>> >> > 2. Permanently keep Airflow Webserver as a separated package
from
> >>> >> Airflow,
> >>> >> > but freeze all development on `www`  and direct all future
UI
> >>> >> development
> >>> >> > to Airflow Webserver, eventually removing `www` completely
when
> >>> Airflow
> >>> >> > Webserver is stable.
> >>> >> > *I'm not a fan of this either. First of all, the views and
models
> >>> are
> >>> >> > tightly coupled in both old and new UI; until we have a
> full-fledged
> >>> >> REST
> >>> >> > API to build the UI (and cli) on top of it, separating them
to a
> >>> >> separate
> >>> >> > package now will potentially cause dependency issues and add
> >>> >> complication
> >>> >> > to our release cycle. **Secondly, **majority of Airflow users
run
> >>> >> Airflow
> >>> >> > with the UI; it's one of Airflow's best features. Separating
UI
> out
> >>> of
> >>> >> > Airflow core will complicate setup and configuration, while
making
> >>> >> Airflow
> >>> >> > core less complete.*
> >>> >> >
> >>> >> > 3. Merge Airflow Webserver back into Airflow as `www2`, freeze
all
> >>> >> > development on `www`, eventually removing `www` completely
when
> >>> `www2`
> >>> >> is
> >>> >> > stable.
> >>> >> > *This makes the most sense to me. Airflow Webserver is developed
> >>> with
> >>> >> the
> >>> >> > goal of feature parity to the current UI, plus additional
RBAC
> >>> >> capability,
> >>> >> > in hope to replace the old UI completely. Yes, this means
there
> >>> will be
> >>> >> a
> >>> >> > short period of having to maintain two UIs, but once we freeze
> >>> >> development
> >>> >> > on www, it shouldn't be a concern for long.*
> >>> >> >
> >>> >> > I'd love to hear everyone's thoughts on this! I'm excited
about
> >>> bringing
> >>> >> > RBAC to airflow and I hope it's something others will find
useful
> as
> >>> >> well!
> >>> >> >
> >>> >> > Cheers,
> >>> >> > Joy
> >>> >> >
> >>> >> > On Mon, Nov 20, 2017 at 11:24 AM, Joy Gao <joyg@wepay.com>
wrote:
> >>> >> >
> >>> >> >> Thank you everyone for the active feedback so far, and
thanks for
> >>> >> setting
> >>> >> >> up the demo Maxime!
> >>> >> >>
> >>> >> >> Going to work on pruning through the issues in the upcoming
days.
> >>> >> >>
> >>> >> >> Fokko/Maxime, do you recall the SQLAlchemy Exception message
so I
> >>> can
> >>> >> >> look into it? Otherwise I'll wait until it's down again
=P
> >>> >> >>
> >>> >> >> Cheers,
> >>> >> >>
> >>> >> >> Joy
> >>> >> >>
> >>> >> >> On Mon, Nov 20, 2017 at 9:35 AM, Maxime Beauchemin <
> >>> >> >> maximebeauchemin@gmail.com> wrote:
> >>> >> >>
> >>> >> >>> I just restarted it, not sure how long it will take
to get in a
> >>> bad
> >>> >> state
> >>> >> >>> again...
> >>> >> >>>
> >>> >> >>> Max
> >>> >> >>>
> >>> >> >>> On Sun, Nov 19, 2017 at 11:55 PM, Driesprong, Fokko
> >>> >> <fokko@driesprong.frl
> >>> >> >>> >
> >>> >> >>> wrote:
> >>> >> >>>
> >>> >> >>> > Good morning,
> >>> >> >>> >
> >>> >> >>> > The demo provided by Max is down, it throws a
> >>> SQLAlchemyexception
> >>> >> :'(
> >>> >> >>> >
> >>> >> >>> > Cheers, Fokko
> >>> >> >>> >
> >>> >> >>> > 2017-11-18 19:14 GMT+01:00 Chris Riccomini <
> >>> criccomini@apache.org>:
> >>> >> >>> >
> >>> >> >>> > > @bolke, open issues on the Github repo,
please.
> >>> >> >>> > >
> >>> >> >>> > > On Sat, Nov 18, 2017 at 10:13 AM, Bolke
de Bruin <
> >>> >> bdbruin@gmail.com>
> >>> >> >>> > > wrote:
> >>> >> >>> > >
> >>> >> >>> > > > Chris,
> >>> >> >>> > > >
> >>> >> >>> > > > Do you want us to report bugs somewhere
(I have
> encountered
> >>> a
> >>> >> >>> few)? Or
> >>> >> >>> > > > just generic user experiences posted
here?
> >>> >> >>> > > >
> >>> >> >>> > > > Cheers
> >>> >> >>> > > > Bolke
> >>> >> >>> > > >
> >>> >> >>> > > > > On 18 Nov 2017, at 00:47, Chris
Riccomini <
> >>> >> criccomini@apache.org
> >>> >> >>> >
> >>> >> >>> > > wrote:
> >>> >> >>> > > > >
> >>> >> >>> > > > > Hey all,
> >>> >> >>> > > > >
> >>> >> >>> > > > > I know the weekend is coming up,
and for those of us in
> >>> the
> >>> >> US,
> >>> >> >>> next
> >>> >> >>> > > week
> >>> >> >>> > > > > is a bit of a slow holiday week.
Would love to get some
> >>> >> feedback
> >>> >> >>> from
> >>> >> >>> > > > > everyone on this. The goal would
ideally to be to
> >>> converge on
> >>> >> >>> this
> >>> >> >>> > and
> >>> >> >>> > > > > eventually replace the existing
Airflow UI with this
> one.
> >>> >> >>> > > > >
> >>> >> >>> > > > > Cheers,
> >>> >> >>> > > > > Chris
> >>> >> >>> > > > >
> >>> >> >>> > > > > On Fri, Nov 17, 2017 at 1:44 PM,
Joy Gao <
> joyg@wepay.com>
> >>> >> wrote:
> >>> >> >>> > > > >
> >>> >> >>> > > > >> Hi guys.
> >>> >> >>> > > > >>
> >>> >> >>> > > > >> I've been working on moving
airflow from Flask-Admin to
> >>> >> >>> > > Flask-AppBuilder
> >>> >> >>> > > > >> for RBAC
> >>> >> >>> > > > >> <https://cwiki.apache.org/confluence/display/AIRFLOW/
> >>> >> >>> > > > Airflow+RBAC+proposal
> >>> >> >>> > > > >>> ,
> >>> >> >>> > > > >> check it out at https://github.com/wepay/airfl
> >>> ow-webserver.
> >>> >> >>> > > > >>
> >>> >> >>> > > > >> It's still a work-in-progress,
but most features you
> see
> >>> in
> >>> >> the
> >>> >> >>> > > > webserver
> >>> >> >>> > > > >> UI today is available there.
For those who are
> >>> interested in
> >>> >> >>> RBAC,
> >>> >> >>> > I'd
> >>> >> >>> > > > love
> >>> >> >>> > > > >> to get some early feedback
in terms of the following:
> >>> >> >>> > > > >>
> >>> >> >>> > > > >> - New Flask-AppBuilder UI
(any bugs/regressions)
> >>> >> >>> > > > >> - Setup issues
> >>> >> >>> > > > >> - Ease of integration with
third party auth (i.e. LDAP,
> >>> AD,
> >>> >> >>> OAuth,
> >>> >> >>> > > etc.)
> >>> >> >>> > > > >> - Any other thoughts/concerns
> >>> >> >>> > > > >>
> >>> >> >>> > > > >> Thanks a lot!
> >>> >> >>> > > > >>
> >>> >> >>> > > > >> Cheers,
> >>> >> >>> > > > >> Joy
> >>> >> >>> > > > >>
> >>> >> >>> > > >
> >>> >> >>> > > >
> >>> >> >>> > >
> >>> >> >>> >
> >>> >> >>>
> >>> >> >>
> >>> >> >>
> >>> >> >
> >>> >>
> >>> >
> >>> >
> >>>
> >>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message