airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Update status on getting to RC5 - reviewers wanted
Date Mon, 06 Mar 2017 16:52:21 GMT
Hi,

Just wanted to do a short update on RC5 status. We had 8 (!) blockers, but the good news they
are either fixed or a patch is available. The patches are in need of a review, so it would
be appreciated if some of the committers can make some time available to do so. Please note
that while reviewing, build errors are expected due to ldap3 and psycopg2 lib issues. In case
no build is available due to throttling on Travis, please check on the user’s GitHub/Travis.
Usually, it is available there.

To be reviewed:

[AIRFLOW-931] Do not set QUEUED in TaskInstances <https://github.com/apache/incubator-airflow/pull/2127>
(https://github.com/apache/incubator-airflow/pull/2127). This fixes with tasks getting stuck
when concurrency has been reached. Might also solve sporadic double triggering of tasks.
[AIRFLOW-941] Use defined parameters for psycopg2 <https://github.com/apache/incubator-airflow/pull/2126>
(https://github.com/apache/incubator-airflow/pull/2126). Work around regression in psycopg2
2.7.0
[AIRFLOW-932] Do not mark tasks removed when backfilling <https://github.com/apache/incubator-airflow/pull/2122>
(https://github.com/apache/incubator-airflow/pull/2122). Make sure not to remove tasks when
only seeing a sub set of tasks of a DAG.
[AIRFLOW-910] Use parallel task execution in backfills <https://github.com/apache/incubator-airflow/pull/2107>
(https://github.com/apache/incubator-airflow/pull/2107). Make task execution concurrent in
backfills and make runs deterministic removing sporadic deadlocks.
[AIRFLOW-900] Double triggered task should not kill original task <https://github.com/apache/incubator-airflow/pull/2102>
(https://github.com/apache/incubator-airflow/pull/2102). Kill the right task when we are double
triggering.

Fixed:
* Skipped operations make DAG finish prematurely
* Setting a task to running manually breaks a DAGs UI
* (Named)HivePartitionSensor broken if hook attr not set
* Can't mark non-existent tasks as successful from graph view


Cheers!
Bolke
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message