airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-5608) Scheduler not shutting down executor(s) gracefully
Date Thu, 17 Oct 2019 13:33:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-5608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953742#comment-16953742
] 

ASF GitHub Bot commented on AIRFLOW-5608:
-----------------------------------------

ashb commented on pull request #6357: Revert "AIRFLOW-5608: Fix bug in SchedulerJob when calling
executor.end"
URL: https://github.com/apache/airflow/pull/6357
 
 
   Reverts apache/airflow#6274
   
   It caused the LocalExecutor to not shutdown when given a SIGINT with this stack trace.
   
   ```
   ^CTraceback (most recent call last):
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 1309,
in _execute
       self._execute_helper()
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 1435,
in _execute_helper
       sleep(sleep_length)
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/cli.py", line 84, in sigint_handler
       sys.exit(0)
   SystemExit: 0
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
     File "/Users/ash/.virtualenvs/airflow/bin/airflow", line 7, in <module>
       exec(compile(f.read(), __file__, 'exec'))
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/airflow", line 39, in <module>
       args.func(args)
     File "/Users/ash/code/python/incubator-airflow/airflow/utils/cli.py", line 73, in wrapper
       return f(*args, **kwargs)
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/cli.py", line 1124, in scheduler
       job.run()
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/base_job.py", line 217, in
run
       self._execute()
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 1314,
in _execute
       self.executor.end()
     File "/Users/ash/code/python/incubator-airflow/airflow/executors/local_executor.py",
line 229, in end
       self.impl.end()
     File "/Users/ash/code/python/incubator-airflow/airflow/executors/local_executor.py",
line 205, in end
       self.queue.put((None, None))
     File "<string>", line 2, in put
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py",
line 796, in _callmethod
       kind, result = conn.recv()
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py",
line 250, in recv
       buf = self._recv_bytes()
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py",
line 407, in _recv_bytes
       buf = self._recv(4)
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py",
line 383, in _recv
       raise EOFError
   EOFError
   ```
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Scheduler not shutting down executor(s) gracefully
> --------------------------------------------------
>
>                 Key: AIRFLOW-5608
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5608
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.5
>            Reporter: Kiran
>            Assignee: Kiran
>            Priority: Major
>             Fix For: 1.10.6
>
>
> If SIGTERM is sent to scheduler, the bug is preventing it from calling graceful shutdown
of executor(s).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message