airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Weise <>
Subject Airflow REST API and task recovery
Date Tue, 28 Mar 2017 15:23:54 GMT

The Airflow documentation mentions an experimental REST API, but I could
not find much further information about it.

I'm looking for a way to use the Airflow functionality through a different
front-end, including:

* Create DAG
* Launch DAG
* Query progress of running DAG (which tasks are active and status of
individual tasks)

Also I was wondering about the fault-tolerance support for operators:

I want to build an operator that submits jobs to the cluster, obtains the
job/application ID and subsequently queries the job status through our own
front-end. Logically each job is one task. If the Airflow worker fails and
recovers, how can the task be recovered (it could have meanwhile changed
status on the cluster)?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message