airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shaw, Damian P. " <damian.sha...@credit-suisse.com>
Subject Manual raising of AirflowRescheduleException supported?
Date Wed, 08 May 2019 15:04:10 GMT
Hi all,

Reading the reschedule code I notice that the execute method of the BaseSensorOperator raises
"AirflowRescheduleException(reschedule_date)" I have a scenario where I would like to reschedule
to a custom datetime, is it supported to manually raise this from my own custom Sensor?

This is the scenario I have: some tasks initially need to wait anywhere from 30 minutes to
12 hours.

Currently I have my own version of the TimeSensor set-up to reschedule itself every 15 minutes
by default and check if the task is ready to run. However this leads to a few problems:

1.       The reschedule drifts away from the 15 minute mark

2.       15 minutes is already a lot more course than I would like

3.       It's still running a process and checking every 15 minutes for up to 12 hours

Thanks for any info,
Damian Shaw

=============================================================================== 
Please access the attached hyperlink for an important electronic communications disclaimer:

http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
=============================================================================== 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message