airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Merging the experimental API Framework
Date Mon, 28 Nov 2016 19:35:07 GMT
There is some work on the wiki. However, I found out that it needs quite a lot of experimenting
to found out the right way to approach things. Hence marking this as experimental. I think
it needs a bit more experimenting, before we get to a stabilized API design doc. I’m also
asking the community for input (code!) as that takes it forward easier imho.

Bolke

> Op 28 nov. 2016, om 20:30 heeft siddharth anand <sanand@apache.org> het volgende
geschreven:
> 
> Bolke,
> Thanks for kicking this off.
> 
> Is there already a design document for this? If not, can you create one? It
> makes sense to have a design document for this to connect multiple PRs. You
> can also add the information above to the same wiki -- The mailing list is
> not always super friendly for historical referencing.
> 
> -s
> 
> On Mon, Nov 28, 2016 at 11:25 AM, Dan Davydov <
> dan.davydov@airbnb.com.invalid> wrote:
> 
>> Just wanted to say this is very exciting, thank you Bolke :).
>> 
>> On Mon, Nov 28, 2016 at 10:50 AM, Bolke de Bruin <bdbruin@gmail.com>
>> wrote:
>> 
>>> All,
>>> 
>>> After a few weeks of work I have finalized the implementation of a Rest
>>> API Framework. Out of the box it supports Kerberos authentication, which
>> is
>>> now fully end to end tested on Travis’ with a working KDC. You can also
>>> switch the CLI to use the API endpoints when available. Currently, only
>> the
>>> “trigger_dag” functionality is available this way, but I hope others to
>>> pick up and create new endpoints that the CLI can then use.
>>> 
>>> For Contributors:
>>> 
>>> In case you are implementing new functionality in the CLI please make
>> sure
>>> to implement the actual functionality in api/common/…/<function_name.py>
>>> and expose it through api_client (abstract), json_client (JSON),
>>> local_client (direct). Endpoints are defined in www/api/experimental.
>>> 
>>> Direct exposure in cli.py I would consider deprecated and I would prefer
>>> to deny it from now on. Hopefully, this gives us a gradual path to
>> improved
>>> integration and improved security while maintaining backwards
>>> compatibility. Also note that the APIs are still marked experimental and
>>> are subject to change.
>>> 
>>> Next steps:
>>> - Swagger definitions (http://swagger.io)
>>> - Research possible integration between different authentication backends
>>> - Use “airflow api” instead of “airflow webserver” to separate concerns
>>> - Remove all direct DB access from cli.py
>>> - Improve documentation
>>> - Design API graduation roadmap (when is something not experimental
>>> anymore)
>>> 
>>> Feedback obviously appreciated.
>>> 
>>> Bolke
>>> 
>>> 
>>> 
>> 


Mime
View raw message