airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-937) task_stats makes extremely large prepared query
Date Thu, 02 Mar 2017 22:05:45 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893114#comment-15893114
] 

ASF subversion and git services commented on AIRFLOW-937:
---------------------------------------------------------

Commit ae44dddcb6cdba6f4e4fa3fe56c3b87e0cd2e502 in incubator-airflow's branch refs/heads/master
from [~saguziel]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=ae44ddd ]

[AIRFLOW-937] Improve performance of task_stats

Please accept this PR that addresses the following
issues:
-
https://issues.apache.org/jira/browse/AIRFLOW-937

Testing Done:
- Shouldn't change functionality significantly,
should pass existing tests (if they exist)

This leads to slightly different results, but it
reduced the time of this endpoint from 90s to 9s
on our data, and the existing logic for task_ids
was already incorrect (task_ids may not be
distinct across dags)

Closes #2121 from saguziel/task-stats-fix


> task_stats makes extremely large prepared query
> -----------------------------------------------
>
>                 Key: AIRFLOW-937
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-937
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Alex Guziel
>            Assignee: Alex Guziel
>
> Right now, the task_stats endpoint makes a few extremely long queries. We can give up
some accuracy and get huge speed wins



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message