airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Proposal to remove json_client
Date Fri, 01 Feb 2019 13:28:40 GMT
The airlow cli can function as this rest client and does not need to be at the same server
as where airflow is running. 

Direct DB access is bad from a separation of concerns perspective as you can change task statusses,
insert arbitrary things etc. 

B.

Sent from my iPhone

> On 1 Feb 2019, at 13:44, David Cavaletto <da.cavaletto@gmail.com> wrote:
> 
> Actually the opposite. If we're going to have a REST API, users should
> interact with it over http(s), using a rest client. If a user has SSH
> access to the server running airflow I don't see the security concern of
> having the CLI access the DB in the same manner the REST API does.
> 
> In my diagram all DB be access would be in the API/common/experimental/*.py
> files, with both REST and CLI APIs making calls to this internal API.
> 
> Can you explain further the security concern with the CLI?
> 
> -Dave
> 
> 
>> On Fri, Feb 1, 2019, 00:24 Bolke de Bruin <bdbruin@gmail.com> wrote:
>> 
>> Hi David,
>> 
>> I assume that you then want to ensure that all calls from the CLI are
>> being made through the Rest-API. In other words the json_client would be
>> the only client remaining. The local_client is a security problem as it
>> needs direct database access which makes it problematic to have normals
>> user use the cli.
>> 
>> If this is the case then yes this would be great, but please note that
>> airflow backfills are more difficult to factor out then.
>> 
>> Cheers
>> Bolke
>> 
>> Verstuurd vanaf mijn iPad
>> 
>>> Op 1 feb. 2019 om 03:26 heeft David Cavaletto <da.cavaletto@gmail.com>
>> het volgende geschreven:
>>> 
>>> In case my images didn't come through, here is a link that shows what
>> I'd like to remove (in red) and add (in green)
>>> 
>>> 
>> https://docs.google.com/drawings/d/1Ux8qGQUdRp2L6YWgqayliIzFiJv1nwev4JaG2rR7oiQ/edit?usp=sharing
>>> 
>>> 
>>>> On Thu, Jan 31, 2019 at 8:21 PM David Cavaletto <da.cavaletto@gmail.com>
>> wrote:
>>>> Hello,
>>>> 
>>>> While trying to refactor the Connection API (both REST and CLI) I've
>> discovered the CLI has the ability to make REST API calls. The flow looks
>> like this:
>>>> 
>>>> 
>>>> In a discussion with Ash in Slack, he said this architecture is so that
>> a user can use a local installation of Airflow (like on their laptop) to
>> issue CLI commands to a remote installation of Airflow. Examples would
>> include a cluster running Fargate, where docker exec and ssh are not
>> available, and so no CLI commands can be run on the installation.
>>>> 
>>>> I'd like to propose that we simplify this entire structure, removing
>> the boxes highlighted in red, like this:
>>>> 
>>>> 
>>>> Users needing to issue REST commands to a cluster would use a REST
>> client (Postman, curl, etc) to issue the commands to the cluster, allowing
>> us to simplify this implementation of the APIs in Airflow.
>>>> 
>>>> Thoughts? Is there some other use case I've overlooked?
>>>> 
>>>> -Dave
>> 

Mime
View raw message