airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mubin Khalid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1147) airflow scheduler not working
Date Fri, 28 Apr 2017 10:32:04 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988609#comment-15988609
] 

Mubin Khalid commented on AIRFLOW-1147:
---------------------------------------

[~g.toonstra] I did create an new test dag, put interval 2 minutes, and it worked fine.
I executed main dag using backfill, it executed successfully, So now I do have last execution
time, right?
so let's say, I executed the dag on 2017-04-27 03:00:00 and on 2017-04-28 03:00:00 it should
be executed automatically, right?
If I run airflow scheduler on 2017-04-28 06:00:00 still it should be able to execute it right,
logically, 2017-04-27 03:00:00 + schedule interval[24 hours or 1440 minutes] = 2017-04-28
03:00:00, and it should run all the dags meeting that criteria. But it's not executing it
even after 24 hours.

Don't know what is wrong with that, because if I run a scheduler, let's say for 15 minutes
interval, it works fine; but not working for 24 hours scenario.


> airflow scheduler not working
> -----------------------------
>
>                 Key: AIRFLOW-1147
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1147
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: Airflow 1.8
>         Environment: CentOS running on 128 GB ram
>            Reporter: Mubin Khalid
>            Priority: Critical
>              Labels: documentation, newbie
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> I've created some `DAG`s, and I tried to put it on scheduler. I want to run all the tasks
in the DAG after exact 24 hours.
> I tried to do something like this.
> {code}
>     DEFAULT_ARGS        = {
>     'owner'           : 'mubin',
>     'depends_on_past' : False,
>     'start_date'      : datetime(2017, 4, 24, 14, 30),
>     'retries'         : 5,
>     'retry_delay'     : timedetla(1),
>     }
>     SCHEDULE_INTERVAL      = timedelta(minutes=1440)
>     # SCHEDULE_INTERVAL    = timedelta(hours=24)
>     # SCHEDULE_INTERVAL    = timedelta(days=1)
>     dag = DAG('StandardizeDataDag',
>         default_args       = DEFAULT_ARGS,
>         schedule_interval  = SCHEDULE_INTERVAL
>     )
>  {code}   
> I tried to put different intervals, but not any working. However if I try to reset db
 {code} airflow resetdb -y {code}  and then run  {code} airflow initdb {code} , it works for
once. then after that, scheduler isn't able to run it.
> PS.  {code} airflow scheduler {code}  executed from  {code} root {code} 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message