airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From criccom...@apache.org
Subject incubator-airflow git commit: [AIRFLOW-196] Fix bug that exception is not handled in HttpSensor
Date Wed, 01 Jun 2016 03:47:20 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/master df8e331ef -> 92f44f5b5


[AIRFLOW-196] Fix bug that exception is not handled in HttpSensor

Dear Airflow Maintainers,

Please accept this PR that addresses the following issues:
- [*AIRFLOW-196*](https://issues.apache.org/jira/browse/AIRFLOW-196)

If exception happens in poke function in HttpSensor, it is not
well handled that make the sensor finish successfully, which is
incorrect obviously.

Author: Junwei Wang <i.junwei.wang@gmail.com>

Closes #1561 from junwei-wang/master.


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/92f44f5b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/92f44f5b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/92f44f5b

Branch: refs/heads/master
Commit: 92f44f5b5b47fdedf7e5908f24ee28bf242b4634
Parents: df8e331
Author: Junwei Wang <i.junwei.wang@gmail.com>
Authored: Tue May 31 20:47:04 2016 -0700
Committer: Chris Riccomini <chrisr@wepay.com>
Committed: Tue May 31 20:47:04 2016 -0700

----------------------------------------------------------------------
 airflow/operators/sensors.py |  2 ++
 tests/operators/sensor.py    | 39 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/92f44f5b/airflow/operators/sensors.py
----------------------------------------------------------------------
diff --git a/airflow/operators/sensors.py b/airflow/operators/sensors.py
index 31ef7b3..569b4d7 100644
--- a/airflow/operators/sensors.py
+++ b/airflow/operators/sensors.py
@@ -516,4 +516,6 @@ class HttpSensor(BaseSensorOperator):
             if ae.message.startswith("404"):
                 return False
 
+            raise ae
+
         return True

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/92f44f5b/tests/operators/sensor.py
----------------------------------------------------------------------
diff --git a/tests/operators/sensor.py b/tests/operators/sensor.py
new file mode 100644
index 0000000..45d4b81
--- /dev/null
+++ b/tests/operators/sensor.py
@@ -0,0 +1,39 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import unittest
+
+from airflow.operators import HttpSensor
+from airflow.exceptions import AirflowException
+
+
+class HttpSensorTests(unittest.TestCase):
+
+    def test_poke_exception(self):
+        """
+        Exception occurs in poke function should not be ignored.
+        """
+        def resp_check(resp):
+            raise AirflowException('AirflowException raised here!')
+
+        task = HttpSensor(
+            task_id='http_sensor_poke_exception',
+            http_conn_id='http_default',
+            endpoint='',
+            params={},
+            response_check=resp_check,
+            poke_interval=5)
+        with self.assertRaisesRegexp(AirflowException, 'AirflowException raised here!'):
+            task.execute(None)


Mime
View raw message