airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joy Gao <j...@wepay.com>
Subject Re: RBAC Update
Date Sat, 24 Mar 2018 01:45:09 GMT
Hey guys!

The RBAC UI <https://github.com/apache/incubator-airflow/pull/3015> has
been merged to master. I'm looking forward to early adopters' feedback and
bug reports. I also hope to have more folks helping out with the RBAC UI,
especially with introducing DAG-Level access control, which is a feature
that a lot of people have been asking. If you are interested in helping out
with this effort, let's talk more!

This commit will be in the 1.10.0 release, and we are going to maintain
both UIs simultaneously for a short period of time. Once RBAC UI is stable
and battle-tested, we will deprecate the old UI and eventually remove it
from the repo (around Airflow 2.0.0 or 2.1.0 release). This is to prevent
two UIs from forking into separate paths, as that would become very
difficult to maintain.

Going forward while both UIs are up, if you are making a change to any
files in airflow/www/ (old UI), where applicable, please also make the
change to the airflow/www_rbac/ (new UI). If you rather not make changes in
both UIs, it is recommended that you only make the changes to the RBAC UI,
since that is the one we are maintaining in the long term.

I'm excited that the RBAC UI will be able to bring additional security to
Airflow, and with FAB framework in place we can look into leveraging it for
a unified set of APIs used by both UI and CLI.

Joy



On Thu, Feb 8, 2018 at 11:31 AM, Joy Gao <joyg@wepay.com> wrote:

> Hi folks,
>
> I have a PR <https://github.com/apache/incubator-airflow/pull/3015> out
> for the new UI. I've included instructions on how to test it out in the PR
> description. Looking forward to your feedbacks.
>
> Cheers,
> Joy
>
> On Fri, Dec 1, 2017 at 6:18 PM, Joy Gao <joyg@wepay.com> wrote:
>
>> Thanks for the background info. Would be really awesome for you to have
>> PyPi access :D I'll make the change to have Airflow Webserver's FAB
>> dependency pointing to my fork for the mean time.
>>
>> For folks who are interested in RBAC, I will be giving a talk/demo at the Airflow
>> Meet-Up
>> <https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/244525050/>
>> next Monday. Happy to chat afterwards about it as well :)
>>
>> On Thu, Nov 30, 2017 at 8:36 AM, Maxime Beauchemin <
>> maximebeauchemin@gmail.com> wrote:
>>
>>> A bit of related history here:
>>> https://github.com/dpgaspar/Flask-AppBuilder/issues/399
>>>
>>> On Thu, Nov 30, 2017 at 8:33 AM, Maxime Beauchemin <
>>> maximebeauchemin@gmail.com> wrote:
>>>
>>> > Given I have merge rights on FAB I could probably do another round of
>>> > review and get your PRs through. I would really like to get the main
>>> > maintainer's input on things that touch the core (composite-key
>>> support) as
>>> > he might have concerns/intuitions that we can't know about.
>>> >
>>> > I do not have Pypi access though so I cannot push new releases out. I
>>> > could ask for that.
>>> >
>>> > I've threatened to fork the project before, that's always an option.
>>> I've
>>> > noticed his involvement is sporadic and comes in bursts.
>>> >
>>> > In the meantime, you can have the dependency in Airflow Webserver
>>> pointing
>>> > straight to your fork.
>>> >
>>> > Max
>>> >
>>> > On Wed, Nov 29, 2017 at 7:02 PM, Joy Gao <joyg@wepay.com> wrote:
>>> >
>>> >> I just created a new webserver instance if you haven't gotten a
>>> chance to
>>> >> fiddle around with the new web UI and the RBAC configurations (thanks
>>> >> Maxime for getting started with this earlier!):
>>> >>
>>> >> http://104.209.38.171:8080/
>>> >>
>>> >> Admin Account
>>> >> username: admin
>>> >> password: admin
>>> >>
>>> >> Read-Only Account
>>> >> username: viewer
>>> >> password: password
>>> >>
>>> >>
>>> >> On Wed, Nov 29, 2017 at 2:58 PM, Joy Gao <joyg@wepay.com> wrote:
>>> >>
>>> >> > Hi folks,
>>> >> >
>>> >> > Thanks for all the feedback regarding to the new Airflow Webserver
>>> UI
>>> >> > <https://github.com/wepay/airflow-webserver/>! I've been
actively
>>> >> > addressing all the bugs that were raised on Github. So I want to
>>> take
>>> >> this
>>> >> > opportunity to discuss two issues coming up:
>>> >> >
>>> >> > The first issue is unaddressed PRs in FAB. If these PRs continue
to
>>> stay
>>> >> > unaddressed, RBAC is blocked from making further progress. If this
>>> >> continue
>>> >> > to be an issue, I'm inclined to fork FAB, even though it's not
>>> >> idealistic.
>>> >> >
>>> >> >
>>> >> >    - PR/631 <https://github.com/dpgaspar/Flask-AppBuilder/pull/631>
>>> >> Binary
>>> >> >    column support (merged, unreleased)
>>> >> >    <https://github.com/dpgaspar/Flask-AppBuilder/pull/631>
>>> >> >    - PR/639 <https://github.com/dpgaspar/Flask-AppBuilder/pull/639>
>>> >> Composite
>>> >> >    primary key support (unmerged)
>>> >> >    - PR/655 <https://github.com/dpgaspar/Flask-AppBuilder/pull/655>
>>> >> Form
>>> >> >    prefill support (unmerged)
>>> >> >
>>> >> >
>>> >> > The second issue is an open question about the next step of Airflow
>>> >> > Webserver itself. Here are the 3 potential directions we could
>>> take, and
>>> >> > I've added my thought on each.
>>> >> >
>>> >> > 1. Permanently keep Airflow Webserver as a separated package from
>>> >> Airflow,
>>> >> > and treat it as another UI option. Keep `www` in Airflow. Allow
>>> >> development
>>> >> > on both UIs.
>>> >> > *I'm not a fan of this. When there is an existing UI in Airflow,
>>> most
>>> >> > contributors would prefer to maintain the official version that
is
>>> >> > installed out-of-the-box. **Having a second UI outside of Airflow
>>> will
>>> >> > make maintaining it very difficult, leading to an eventual death
of
>>> the
>>> >> new
>>> >> > UI :(*
>>> >> >
>>> >> > 2. Permanently keep Airflow Webserver as a separated package from
>>> >> Airflow,
>>> >> > but freeze all development on `www`  and direct all future UI
>>> >> development
>>> >> > to Airflow Webserver, eventually removing `www` completely when
>>> Airflow
>>> >> > Webserver is stable.
>>> >> > *I'm not a fan of this either. First of all, the views and models
>>> are
>>> >> > tightly coupled in both old and new UI; until we have a full-fledged
>>> >> REST
>>> >> > API to build the UI (and cli) on top of it, separating them to
a
>>> >> separate
>>> >> > package now will potentially cause dependency issues and add
>>> >> complication
>>> >> > to our release cycle. **Secondly, **majority of Airflow users run
>>> >> Airflow
>>> >> > with the UI; it's one of Airflow's best features. Separating UI
out
>>> of
>>> >> > Airflow core will complicate setup and configuration, while making
>>> >> Airflow
>>> >> > core less complete.*
>>> >> >
>>> >> > 3. Merge Airflow Webserver back into Airflow as `www2`, freeze
all
>>> >> > development on `www`, eventually removing `www` completely when
>>> `www2`
>>> >> is
>>> >> > stable.
>>> >> > *This makes the most sense to me. Airflow Webserver is developed
>>> with
>>> >> the
>>> >> > goal of feature parity to the current UI, plus additional RBAC
>>> >> capability,
>>> >> > in hope to replace the old UI completely. Yes, this means there
>>> will be
>>> >> a
>>> >> > short period of having to maintain two UIs, but once we freeze
>>> >> development
>>> >> > on www, it shouldn't be a concern for long.*
>>> >> >
>>> >> > I'd love to hear everyone's thoughts on this! I'm excited about
>>> bringing
>>> >> > RBAC to airflow and I hope it's something others will find useful
as
>>> >> well!
>>> >> >
>>> >> > Cheers,
>>> >> > Joy
>>> >> >
>>> >> > On Mon, Nov 20, 2017 at 11:24 AM, Joy Gao <joyg@wepay.com>
wrote:
>>> >> >
>>> >> >> Thank you everyone for the active feedback so far, and thanks
for
>>> >> setting
>>> >> >> up the demo Maxime!
>>> >> >>
>>> >> >> Going to work on pruning through the issues in the upcoming
days.
>>> >> >>
>>> >> >> Fokko/Maxime, do you recall the SQLAlchemy Exception message
so I
>>> can
>>> >> >> look into it? Otherwise I'll wait until it's down again =P
>>> >> >>
>>> >> >> Cheers,
>>> >> >>
>>> >> >> Joy
>>> >> >>
>>> >> >> On Mon, Nov 20, 2017 at 9:35 AM, Maxime Beauchemin <
>>> >> >> maximebeauchemin@gmail.com> wrote:
>>> >> >>
>>> >> >>> I just restarted it, not sure how long it will take to
get in a
>>> bad
>>> >> state
>>> >> >>> again...
>>> >> >>>
>>> >> >>> Max
>>> >> >>>
>>> >> >>> On Sun, Nov 19, 2017 at 11:55 PM, Driesprong, Fokko
>>> >> <fokko@driesprong.frl
>>> >> >>> >
>>> >> >>> wrote:
>>> >> >>>
>>> >> >>> > Good morning,
>>> >> >>> >
>>> >> >>> > The demo provided by Max is down, it throws a
>>> SQLAlchemyexception
>>> >> :'(
>>> >> >>> >
>>> >> >>> > Cheers, Fokko
>>> >> >>> >
>>> >> >>> > 2017-11-18 19:14 GMT+01:00 Chris Riccomini <
>>> criccomini@apache.org>:
>>> >> >>> >
>>> >> >>> > > @bolke, open issues on the Github repo, please.
>>> >> >>> > >
>>> >> >>> > > On Sat, Nov 18, 2017 at 10:13 AM, Bolke de Bruin
<
>>> >> bdbruin@gmail.com>
>>> >> >>> > > wrote:
>>> >> >>> > >
>>> >> >>> > > > Chris,
>>> >> >>> > > >
>>> >> >>> > > > Do you want us to report bugs somewhere
(I have encountered
>>> a
>>> >> >>> few)? Or
>>> >> >>> > > > just generic user experiences posted here?
>>> >> >>> > > >
>>> >> >>> > > > Cheers
>>> >> >>> > > > Bolke
>>> >> >>> > > >
>>> >> >>> > > > > On 18 Nov 2017, at 00:47, Chris Riccomini
<
>>> >> criccomini@apache.org
>>> >> >>> >
>>> >> >>> > > wrote:
>>> >> >>> > > > >
>>> >> >>> > > > > Hey all,
>>> >> >>> > > > >
>>> >> >>> > > > > I know the weekend is coming up, and
for those of us in
>>> the
>>> >> US,
>>> >> >>> next
>>> >> >>> > > week
>>> >> >>> > > > > is a bit of a slow holiday week. Would
love to get some
>>> >> feedback
>>> >> >>> from
>>> >> >>> > > > > everyone on this. The goal would ideally
to be to
>>> converge on
>>> >> >>> this
>>> >> >>> > and
>>> >> >>> > > > > eventually replace the existing Airflow
UI with this one.
>>> >> >>> > > > >
>>> >> >>> > > > > Cheers,
>>> >> >>> > > > > Chris
>>> >> >>> > > > >
>>> >> >>> > > > > On Fri, Nov 17, 2017 at 1:44 PM, Joy
Gao <joyg@wepay.com>
>>> >> wrote:
>>> >> >>> > > > >
>>> >> >>> > > > >> Hi guys.
>>> >> >>> > > > >>
>>> >> >>> > > > >> I've been working on moving airflow
from Flask-Admin to
>>> >> >>> > > Flask-AppBuilder
>>> >> >>> > > > >> for RBAC
>>> >> >>> > > > >> <https://cwiki.apache.org/confluence/display/AIRFLOW/
>>> >> >>> > > > Airflow+RBAC+proposal
>>> >> >>> > > > >>> ,
>>> >> >>> > > > >> check it out at https://github.com/wepay/airfl
>>> ow-webserver.
>>> >> >>> > > > >>
>>> >> >>> > > > >> It's still a work-in-progress,
but most features you see
>>> in
>>> >> the
>>> >> >>> > > > webserver
>>> >> >>> > > > >> UI today is available there. For
those who are
>>> interested in
>>> >> >>> RBAC,
>>> >> >>> > I'd
>>> >> >>> > > > love
>>> >> >>> > > > >> to get some early feedback in terms
of the following:
>>> >> >>> > > > >>
>>> >> >>> > > > >> - New Flask-AppBuilder UI (any
bugs/regressions)
>>> >> >>> > > > >> - Setup issues
>>> >> >>> > > > >> - Ease of integration with third
party auth (i.e. LDAP,
>>> AD,
>>> >> >>> OAuth,
>>> >> >>> > > etc.)
>>> >> >>> > > > >> - Any other thoughts/concerns
>>> >> >>> > > > >>
>>> >> >>> > > > >> Thanks a lot!
>>> >> >>> > > > >>
>>> >> >>> > > > >> Cheers,
>>> >> >>> > > > >> Joy
>>> >> >>> > > > >>
>>> >> >>> > > >
>>> >> >>> > > >
>>> >> >>> > >
>>> >> >>> >
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >
>>> >>
>>> >
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message