airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Imberman (Jira)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-5447) KubernetesExecutor hangs on task queueing
Date Mon, 16 Sep 2019 18:19:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930754#comment-16930754
] 

Daniel Imberman commented on AIRFLOW-5447:
------------------------------------------

[~cwegrzyn] this is nuts... It might be a race condition/failure in multiprocessing. How
many tasks are you trying to launch at once? This is giving me a lot to investigate with btw
so thank you.

> KubernetesExecutor hangs on task queueing
> -----------------------------------------
>
>                 Key: AIRFLOW-5447
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5447
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: executor-kubernetes
>    Affects Versions: 1.10.4, 1.10.5
>         Environment: Kubernetes version v1.14.3, Airflow version 1.10.4-1.10.5
>            Reporter: Henry Cohen
>            Assignee: Daniel Imberman
>            Priority: Blocker
>
> Starting in 1.10.4, and continuing in 1.10.5, when using the KubernetesExecutor, with
the webserver and scheduler running in the kubernetes cluster, tasks are scheduled, but when
added to the task queue, the executor process hangs indefinitely. Based on log messages, it
appears to be stuck at this line https://github.com/apache/airflow/blob/v1-10-stable/airflow/contrib/executors/kubernetes_executor.py#L761



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message