airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Weise <...@apache.org>
Subject Airflow REST API and task recovery
Date Tue, 28 Mar 2017 15:23:54 GMT
Hi,

The Airflow documentation mentions an experimental REST API, but I could
not find much further information about it.

I'm looking for a way to use the Airflow functionality through a different
front-end, including:

* Create DAG
* Launch DAG
* Query progress of running DAG (which tasks are active and status of
individual tasks)

Also I was wondering about the fault-tolerance support for operators:

I want to build an operator that submits jobs to the cluster, obtains the
job/application ID and subsequently queries the job status through our own
front-end. Logically each job is one task. If the Airflow worker fails and
recovers, how can the task be recovered (it could have meanwhile changed
status on the cluster)?

Thanks!
Thomas

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message