airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-2664) Support filtering on /dags/<dag_id>/dag_runs
Date Sun, 02 Sep 2018 18:00:04 GMT


Apache Spark commented on AIRFLOW-2664:

User 'rcorre' has created a pull request for this issue:

> Support filtering on /dags/<dag_id>/dag_runs
> --------------------------------------------
>                 Key: AIRFLOW-2664
>                 URL:
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: api
>            Reporter: Ryan Roden-Corrent
>            Priority: Major
> the `GET /dags/<dag_id>/dag_runs` endpoint was recently added as described in AIRFLOW-2550.
I'd like a ?run_id_prefix parameter to filter runs by the run_id.
> For example, `GET /dags/my_dag/dag_runs?run_id_prefix=create_namespace_foo would return
the list of dag runs with a run_id starting with "create_namespace_foo".
> An additional filter would be a `?latest=<n>` query param to limit the count to
the latest <n> runs (I believe the list returned by GET is already ordered chronologically).
For example, `GET /dags/my_dag/dag_runs?run_id_prefix=create_namespace_foo&latest=1` would
return the last run with run_id starting with create_namespace_foo.
> Do these features sound generally useful, or are they too specific?
> In our use case, we have a DAG that provisions namespaced resources. If I'm provisioning
resources for namespace foo, I would `POST /dags/provision/dag_runs \{"run_id": "create_namespace_foo"}`.
To check on the status, we could run `GET /dags/provision/dag_runs?run_id_prefix=create_namespace_foo&latest=1`.
> While there is already an endpoint to get status based on an exact dag_run ID, this requires
storing the id after the POST, which would make our service stateful. Filtering by a prefix
allows for stateless querying of status.

This message was sent by Atlassian JIRA

View raw message