airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Song Liu <>
Subject 答复: About multi user support in airflow
Date Fri, 02 Feb 2018 04:31:25 GMT
Hi Trent,

Thanks for your sharing.


You can filter the list of dags in webserver by owner name when authentication is turned on
by setting webserver:filter_by_owner in your config. With this, a user will see only the dags
which it is owner of, unless it is a superuser.

filter_by_owner = True

It seems that it could be achieved by the filter_by_owner, I would try is this usable already
or not.

But here I still have another question about the dags management:

1- for the cluster deployment, what's the better way to sync among the scheduler and other
workers ? and why not build-in this sync mechanism ?
2- one real case that when I add a new dag into dags folder, I need refresh many times via.
web UI to see it and it will appear the notice about "scheduler seems doesn't see this dag
locally", I want to know when a new dag added, how does the scheduler will do ? will it parse
the new added dag only ?

Thanks again.

发件人: Trent Robbins <>
发送时间: 2018年2月2日 4:19
收件人: Song Liu
主题: Re: About multi user support in airflow

Hi Song,

It looks like there is a basic "owner" DAG filter feature already available but I do not know
how detailed the implementation is:

I can't speak for the developers but I assume that a fully built out feature hasn't been an
extremely high priority for anyone or perhaps there is more code support than I know about
and it isn't fully documented.

From a quick scan of the project I see that flask-login is installed during setup although
that doesn't necessarily mean that it plays a role in these features.


On Thu, Feb 1, 2018 at 20:12 Song Liu <<>>
Hi Trent,

One way is to deploy one standalone airflow components (scheduler, worker, broker, mysql etc.)
in docker for every single user, but there needs to only one web service for user accessing,
that would mean the web service should be interact with multiple other worker / scheduler
for individual user, do I understand correctly ?

For Flask, how could the multi-user feature be supported ? could you help clarify ?

Also, could you help share what's the reason/design behind why it doesn't support multi-user
feature in airflow ?

发件人: Trent Robbins <<>>
发送时间: 2018年2月2日 3:50
主题: Re: About multi user support in airflow

Hi Song,

I would recommend using airflow in docker with individual databases for
individual contributors. Does this make sense for your needs? There are
other similar concepts that could also be used.

I don't think there is another way to split DAGs by user permission.
However, this feature doesn't seem particularly challenging to implement
since it uses Flask. But who knows!


On Thu, Feb 1, 2018 at 19:24 Song Liu <<>>

> Hi,
> In production, multi-user support is needed, which means that every user
> could login the airflow platform, and could manage their own dags
> separately.
> But currently it seem that all the dags are managed into a single dags
> folder, so for multi-user support what I need to do ?
> Many thanks for helping out.
> Thanks,
> Song
(Sent from cellphone)
(Sent from cellphone)
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message