airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mubin Khalid (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-1147) airflow scheduler not working
Date Tue, 25 Apr 2017 19:46:04 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mubin Khalid updated AIRFLOW-1147:
----------------------------------
    Description: 
I've created some `DAG`s, and I tried to put it on scheduler. I want to run all the tasks
in the DAG after exact 24 hours.
I tried to do something like this.
{code}
    DEFAULT_ARGS        = {
    'owner'           : 'mubin',
    'depends_on_past' : False,
    'start_date'      : datetime(2017, 4, 24, 14, 30),
    'retries'         : 5,
    'retry_delay'     : timedetla(1),
    }
    SCHEDULE_INTERVAL      = timedelta(minutes=1440)
    # SCHEDULE_INTERVAL    = timedelta(hours=24)
    # SCHEDULE_INTERVAL    = timedelta(days=1)
    dag = DAG('StandardizeDataDag',
        default_args       = DEFAULT_ARGS,
        schedule_interval  = SCHEDULE_INTERVAL
    )
 {code}   
I tried to put different intervals, but not any working. However if I try to reset db  {code}
airflow resetdb -y {code}  and then run  {code} airflow initdb {code} , it works for once.
then after that, scheduler isn't able to run it.

PS.  {code} airflow scheduler {code}  executed from  {code} root {code} 


  was:
I've created some `DAG`s, and I tried to put it on scheduler. I want to run all the tasks
in the DAG after exact 24 hours.
I tried to do something like this.
{code}
    DEFAULT_ARGS        = {
    'owner'           : 'mubin',
    'depends_on_past' : False,
    'start_date'      : datetime(2017, 4, 24, 14, 30),
    'retries'         : 5,
    'retry_delay'     : timedetla(1),
    }
    SCHEDULE_INTERVAL      = timedelta(minutes=1440)
    # SCHEDULE_INTERVAL    = timedelta(hours=24)
    # SCHEDULE_INTERVAL    = timedelta(days=1)
    dag = DAG('StandardizeDataDag',
        default_args       = DEFAULT_ARGS,
        schedule_interval  = SCHEDULE_INTERVAL
    )
 {code}   
I tried to put different intervals, but not any working. However if I try to reset db `airflow
resetdb -y` and then run `airflow initdb`, it works for once. then after that, scheduler isn't
able to run it.

PS. `airflow scheduler` is executed via `root`

can anybody point me what I'm doing wrong?


> airflow scheduler not working
> -----------------------------
>
>                 Key: AIRFLOW-1147
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1147
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: Airflow 1.8
>         Environment: CentOS running on 128 GB ram
>            Reporter: Mubin Khalid
>            Priority: Critical
>              Labels: documentation, newbie
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> I've created some `DAG`s, and I tried to put it on scheduler. I want to run all the tasks
in the DAG after exact 24 hours.
> I tried to do something like this.
> {code}
>     DEFAULT_ARGS        = {
>     'owner'           : 'mubin',
>     'depends_on_past' : False,
>     'start_date'      : datetime(2017, 4, 24, 14, 30),
>     'retries'         : 5,
>     'retry_delay'     : timedetla(1),
>     }
>     SCHEDULE_INTERVAL      = timedelta(minutes=1440)
>     # SCHEDULE_INTERVAL    = timedelta(hours=24)
>     # SCHEDULE_INTERVAL    = timedelta(days=1)
>     dag = DAG('StandardizeDataDag',
>         default_args       = DEFAULT_ARGS,
>         schedule_interval  = SCHEDULE_INTERVAL
>     )
>  {code}   
> I tried to put different intervals, but not any working. However if I try to reset db
 {code} airflow resetdb -y {code}  and then run  {code} airflow initdb {code} , it works for
once. then after that, scheduler isn't able to run it.
> PS.  {code} airflow scheduler {code}  executed from  {code} root {code} 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message