airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fokko Driesprong (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (AIRFLOW-1252) Experimental API - exception when conf is present in JSON body
Date Sun, 18 Nov 2018 17:35:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fokko Driesprong resolved AIRFLOW-1252.
---------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0

> Experimental API - exception when conf is present in JSON body
> --------------------------------------------------------------
>
>                 Key: AIRFLOW-1252
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1252
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: api
>    Affects Versions: 1.8.0, 1.8.1, 1.9.0, 1.10.0, 2.0.0
>            Reporter: Sergio Herrera
>            Assignee: Sergio Herrera
>            Priority: Major
>              Labels: api
>             Fix For: 2.0.0
>
>
> When someones calls to the endpoint _POST <host>:<port>/api/experimental/dags/<dag_id>/dag_runs
{}_, Airflow never run that request if the body of that contains _conf_.
>  This occurs due to a mismatch between types when calling function _trigger_dag()_, which
is also used by *CLI*. That function perform a _json.loads(conf)_ because from CLI the type
of conf is _string_, but, in the other side, from *experimental API*, that type is _dict_
(because _Json_ is processed before to get all data, such as execution_date).
> There are two possibilities:
>  1. Look for every use of _trigger_dag()_ function and put _Json_ formatting from outside
the function.
>  2. In the *experimental API*, put the conf in a string (with _json.dumps()_) to allow
_trigger_dag()_ transform into _dict_.
> I have implemented the second option, so I can make a PR with that if you want.
> Thank you a lot
> EDIT: Also, there are no tests which uses conf in the Json passed through request currently.
> Examples:
>  - Before fix (escaped json):
> {noformat}
> POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1
> Content-Type: application/json
> {
>   "conf": "{
>     \"k1\": \"v1\",
>     \"k2\": \"v2\",
>     \"k3\": [\"av1\", \"av2\", \"av3\"],
>     \"k4\": {
>       \"sk1\": \"sv1\",
>       \"sk2\": \"sv2\"
>     }
>   }"  
> }
> {noformat}
>  - After fix (pure json):
> {noformat}
> POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1
> Content-Type: application/json
> {
>   "conf": {
>     "k1": "v1",
>     "k2": "v2",
>     "k3": ["av1", "av2", "av3"],
>     "k4": {
>       "sk1": "sv1",
>       "sk2": "sv2"
>     }
>   }
> }
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message