airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bo...@apache.org
Subject incubator-airflow git commit: [AIRFLOW-1745] Restore default signal disposition
Date Sat, 21 Oct 2017 08:25:11 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-9-test 974b493fc -> e021c9d0a


[AIRFLOW-1745] Restore default signal disposition

Restore defaults for SIGPIPE, SIGXFZ, and SIGXFSZ
Python 2.7 subprocess resets signal disposition
for these
signals to ignore, which can cause problems.  For
example,
a simple BashOperator executing 'yes | head' may
never
terminate.  For details, see discussion at:
https://bugs.python.org/issue1652
https://stackoverflow.com/questions/22077881/yes-
reporting-error-with-subprocess-communicate
etc.

Closes #2714 from wrp/sigpipe

(cherry picked from commit ca961042c146d49504e00e4abefc7779f0747782)
Signed-off-by: Bolke de Bruin <bolke@xs4all.nl>


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/e021c9d0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/e021c9d0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/e021c9d0

Branch: refs/heads/v1-9-test
Commit: e021c9d0a4e45465f3702249bf5eadbe40bf9135
Parents: 974b493
Author: William Pursell <williamp@wepay.com>
Authored: Sat Oct 21 10:24:52 2017 +0200
Committer: Bolke de Bruin <bolke@xs4all.nl>
Committed: Sat Oct 21 10:25:06 2017 +0200

----------------------------------------------------------------------
 airflow/operators/bash_operator.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/e021c9d0/airflow/operators/bash_operator.py
----------------------------------------------------------------------
diff --git a/airflow/operators/bash_operator.py b/airflow/operators/bash_operator.py
index ff2ed51..a7eeb03 100644
--- a/airflow/operators/bash_operator.py
+++ b/airflow/operators/bash_operator.py
@@ -79,12 +79,18 @@ class BashOperator(BaseOperator):
                     "Temporary script location: %s",
                     script_location
                 )
+                def pre_exec():
+                    # Restore default signal disposition and invoke setsid
+                    for sig in ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ'):
+                        if hasattr(signal, sig):
+                            signal.signal(getattr(signal, sig), signal.SIG_DFL)
+                    os.setsid()
                 self.log.info("Running command: %s", bash_command)
                 sp = Popen(
                     ['bash', fname],
                     stdout=PIPE, stderr=STDOUT,
                     cwd=tmp_dir, env=self.env,
-                    preexec_fn=os.setsid)
+                    preexec_fn=pre_exec)
 
                 self.sp = sp
 


Mime
View raw message