airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Getting to RC5: Update
Date Wed, 01 Mar 2017 16:22:47 GMT
Hi,

Just wanted to give an update about the progress getting to RC5. As reported we have 6 blockers
listed.

1. Double run job should not terminate the existing running job. -> Patch Available
2. Parallelize dag runs in backfills -> Patch Available, Tests need to be updated, see
below
3. Setting a task to running manually breaks a DAGs UI -> Patch merged
4. Can't mark non-existent tasks as successful from graph view -> Workaround available
(t.b.c.), Patch Available unit tests to be added
5. (Named)HivePartitionSensor broken if hook attr not set -> Patch merged
6. Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely ->
see below

On 2 I would like to have some more discussion of this would be acceptable (https://github.com/apache/incubator-airflow/pull/2107).
I have written the patch for this, however we are not large backfill users. So I need feedback
specifically on ripping out the “executor” part: @dan, @max.

On 6 Alex has reported this earlier and written a PR for this (https://github.com/apache/incubator-airflow/pull/1961).
Maxime had some thoughts about this, which are currently blocking the integration. However,
in testing it seems to solve the issue. Can we finalise the discussion please @max @dan @alex?

Cheers
Bolke
Mime
View raw message