airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From criccom...@apache.org
Subject [01/36] incubator-airflow git commit: [AIRFLOW-974] Fix mkdirs race condition
Date Tue, 09 May 2017 17:35:30 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-8-stable 8e7a55836 -> 0d8509e7e


[AIRFLOW-974] Fix mkdirs race condition

mkdirs congtained a race condition for when if the
directory is
created between the os.path.exists and the
os.makedirs calls,
the os.makedirs will fail with an OSError.

This reworks the function to be non-recursive as
well, as
permission errors were due to umasks being
applied.

Closes #2147 from bolkedebruin/AIRFLOW-974

(cherry picked from commit c5cc298cf16c9777c90aec1fc8cc24bde62f7b2f)
Signed-off-by: Bolke de Bruin <bolke@Bolkes-MacBook-Pro.local>


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/3b37cfa1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/3b37cfa1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/3b37cfa1

Branch: refs/heads/v1-8-stable
Commit: 3b37cfa1f2642ff90908a3af0a5674637c9518ee
Parents: 2a60897
Author: Bolke de Bruin <bolke@xs4all.nl>
Authored: Mon Mar 13 20:14:07 2017 -0700
Committer: Bolke de Bruin <bolke@Bolkes-MacBook-Pro.local>
Committed: Mon Mar 13 20:14:30 2017 -0700

----------------------------------------------------------------------
 airflow/utils/file.py | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/3b37cfa1/airflow/utils/file.py
----------------------------------------------------------------------
diff --git a/airflow/utils/file.py b/airflow/utils/file.py
index 78ddeaa..352755e 100644
--- a/airflow/utils/file.py
+++ b/airflow/utils/file.py
@@ -44,16 +44,14 @@ def mkdirs(path, mode):
 
     :param path: The directory to create
     :type path: str
-    :param mode: The mode to give to the directory e.g. 0o755
+    :param mode: The mode to give to the directory e.g. 0o755, ignores umask
     :type mode: int
-    :return: A list of directories that were created
-    :rtype: list[str]
     """
-    if not path or os.path.exists(path):
-        return []
-    (head, _) = os.path.split(path)
-    res = mkdirs(head, mode)
-    os.mkdir(path)
-    os.chmod(path, mode)
-    res += [path]
-    return res
+    try:
+        o_umask = os.umask(0)
+        os.makedirs(path, mode)
+    except OSError:
+        if not os.path.isdir(path):
+            raise
+    finally:
+        os.umask(o_umask)


Mime
View raw message