airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-3145) [Edit Docummntation ] reduce airflow dag scheduling latency in production
Date Mon, 08 Oct 2018 08:08:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

jack updated AIRFLOW-3145:
--------------------------
    Component/s: docs
        Summary: [Edit Docummntation ] reduce airflow dag scheduling latency in production
 (was: reduce airflow dag scheduling latency in production)

> [Edit Docummntation ] reduce airflow dag scheduling latency in production
> -------------------------------------------------------------------------
>
>                 Key: AIRFLOW-3145
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3145
>             Project: Apache Airflow
>          Issue Type: Task
>          Components: docs
>    Affects Versions: 1.10.1
>            Reporter: jack
>            Priority: Trivial
>
> The docs says:
>  * {{}}
> {code:java}
> max_threads: Scheduler will spawn multiple threads in parallel to schedule dags. This
is controlled by max_threads with default value of 2. User should increase this value to
a larger value(e.g numbers of cpus where scheduler runs - 1) in production.{code}
> [https://airflow.apache.org/faq.html#how-can-my-airflow-dag-run-faster]
>  
>  
> However in the Aiflow.cfg the comment is
> {code:java}
> #The scheduler can run multiple threads in parallel to schedule dags.
> # This defines how many threads will run.
> max_threads = 2{code}
>  
>  
> May I suggest that the
> {code:java}
> User should increase this value to a larger value(e.g numbers of cpus where scheduler
runs - 1) in production.{code}
>  
> will also be written in the comment at Aiflow.cfg 
> Not everyone read the docs from top to bottom this could be easily missed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message