airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Grubb <Aaron.Gr...@clearpier.com>
Subject RE: Taskes not running with celery gevent/eventlet worker
Date Tue, 07 Apr 2020 13:05:45 GMT
I think this is the same issue I encountered. Try running the celery workers manually (i.e.
using the celery command to launch them).

From: Sophie Herrmann <sophie.herrmann@eodc.eu>
Sent: Tuesday, April 7, 2020 8:21 AM
To: users@airflow.apache.org
Subject: Taskes not running with celery gevent/eventlet worker

Hi everyone,

I am using Airflow 1.10.9 with the CeleryExecutor (docker image based on puckel/docker-airflow)
and I am trying to switch from prefork to gevent/eventlet pool for one worker.
With prefork everything is working as expected but when I use a gevent/eventlet worker instead,
it seems tasks are not picked up.

I checked the logs and the scheduler queues the task and sends them to the worker. The worker
then receives the task but after that nothing else happens (execution is not started).
Below the last logs I get from the worker:

[2020-04-07 12:13:13,376: INFO/MainProcess] Received task: airflow.executors.celery_executor.execute_command[32ba869c-80b8-47b5-8eca-aadd64c7fd7c]
[2020-04-07 12:13:13,614: INFO/MainProcess] Scaling up 1 processes.
[2020-04-07 12:13:13,630: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task
at 0x7fb8af20fdd0> (args:('airflow.executors.celery_executor.execute_command', '32ba869c-80b8-47b5-8eca-aadd64c7fd7c',
{'lang': 'py', 'task': 'airflow.executors.celery_executor.execute_command', 'id': '32ba869c-80b8-47b5-8eca-aadd64c7fd7c',
'shadow': None, 'eta': None, 'expires': None, 'group': None, 'retries': 0, 'timelimit': [None,
None], 'root_id': '32ba869c-80b8-47b5-8eca-aadd64c7fd7c', 'parent_id': None, 'argsrepr': "[['airflow',
'run', 'jb-10baee25-8b57-46d1-bb9b-75380504df33', 'cancel_sensor', '2020-04-07T12:13:09.378481+00:00',
'--local', '--pool', 'default_pool', '-sd', '/usr/local/airflow/dags/dag_jb-10baee25-8b57-46d1-bb9b-75380504df33.py']]",
'kwargsrepr': '{}', 'origin': 'gen76@f2822480e988', 'reply_to': 'e07ad7c8-e252-3533-a528-11b3eb712a22',
'correlation_id': '32ba869c-80b8-47b5-8eca-aadd64c7fd7c', 'hostname': 'celery@9ede4f9fb1d1',
'delivery_info': {'exchange': '', 'routing_key': 'sensor', 'priority': 0, 'redelivered': None},
'args': [['airflow', 'run', 'jb-10baee25-8b57-46d1-bb9b-75380504df33', 'cancel_sensor',...
kwargs:{})

In the Airflow UI the task is set to queued and does not change.

Any idea what could be the problem?

Best,
Sophie



Mime
View raw message