airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shaw, Damian P. " <damian.sha...@credit-suisse.com>
Subject RE: Manual raising of AirflowRescheduleException supported?
Date Thu, 09 May 2019 22:35:29 GMT
Thanks Stefan!

Works like a charm. In fact because reschedule_date can be another timezone it's now much
easier to make DAGs where tasks are rescheduled against different timezones. My users are
very happy.

Regards,
Damian

-----Original Message-----
From: Stefan Seelmann [mailto:mail@stefan-seelmann.de] 
Sent: Wednesday, May 08, 2019 11:14 AM
To: dev@airflow.apache.org
Subject: Re: Manual raising of AirflowRescheduleException supported?

Yes, for sensors it should work to raise AirflowRescheduleException,
that was one design goal [1]. I wanted to implement it for TimeSensor
but didn't find time yet.

Kind Regards,
Stefan

[1] https://issues.apache.org/jira/browse/AIRFLOW-2747

On 5/8/19 5:04 PM, Shaw, Damian P.  wrote:
> Hi all,
> 
> Reading the reschedule code I notice that the execute method of the BaseSensorOperator
raises "AirflowRescheduleException(reschedule_date)" I have a scenario where I would like
to reschedule to a custom datetime, is it supported to manually raise this from my own custom
Sensor?
> 
> This is the scenario I have: some tasks initially need to wait anywhere from 30 minutes
to 12 hours.
> 
> Currently I have my own version of the TimeSensor set-up to reschedule itself every 15
minutes by default and check if the task is ready to run. However this leads to a few problems:
> 
> 1.       The reschedule drifts away from the 15 minute mark
> 
> 2.       15 minutes is already a lot more course than I would like
> 
> 3.       It's still running a process and checking every 15 minutes for up to 12 hours
> 
> Thanks for any info,
> Damian Shaw
> 
> =============================================================================== 
> Please access the attached hyperlink for an important electronic communications disclaimer:

> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
> =============================================================================== 
> 



=============================================================================== 
Please access the attached hyperlink for an important electronic communications disclaimer:

http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
=============================================================================== 

Mime
View raw message