airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xua...@apache.org
Subject incubator-airflow git commit: [AIRFLOW-498] Remove hard-coded gcp project id
Date Wed, 14 Sep 2016 12:35:42 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/master 8fba0acab -> fb7b98b63


[AIRFLOW-498] Remove hard-coded gcp project id

Closes #1786 from julianvmodesto/bug/AIRFLOW-498
--fix-gcp-dataflow-hook


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/fb7b98b6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/fb7b98b6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/fb7b98b6

Branch: refs/heads/master
Commit: fb7b98b63663b5214059f805abde105b3e20831e
Parents: 8fba0ac
Author: Julian V. Modesto <julian.modesto@nytimes.com>
Authored: Wed Sep 14 08:35:35 2016 -0400
Committer: Li Xuanji <xuanji@gmail.com>
Committed: Wed Sep 14 08:35:35 2016 -0400

----------------------------------------------------------------------
 airflow/contrib/hooks/gcp_dataflow_hook.py | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/fb7b98b6/airflow/contrib/hooks/gcp_dataflow_hook.py
----------------------------------------------------------------------
diff --git a/airflow/contrib/hooks/gcp_dataflow_hook.py b/airflow/contrib/hooks/gcp_dataflow_hook.py
index 7c35f3f..bd5bd3c 100644
--- a/airflow/contrib/hooks/gcp_dataflow_hook.py
+++ b/airflow/contrib/hooks/gcp_dataflow_hook.py
@@ -121,11 +121,6 @@ class DataFlowHook(GoogleCloudBaseHook):
     def __init__(self,
                  gcp_conn_id='google_cloud_default',
                  delegate_to=None):
-        """
-        :param scope: The scope of the hook (read only, read write, etc). See:
-            https://cloud.google.com/storage/docs/authentication?hl=en#oauth-scopes
-        :type scope: string
-        """
         super(DataFlowHook, self).__init__(gcp_conn_id, delegate_to)
 
     def get_conn(self):
@@ -139,7 +134,7 @@ class DataFlowHook(GoogleCloudBaseHook):
         name = task_id + "-" + str(uuid.uuid1())[:8]
         cmd = self._build_cmd(task_id, variables, dataflow, name)
         _DataflowJava(cmd).wait_for_done()
-        _DataflowJob(self.get_conn(), "vex-eu-cloud-sql-001", name).wait_for_done()
+        _DataflowJob(self.get_conn(), variables['project'], name).wait_for_done()
 
     def _build_cmd(self, task_id, variables, dataflow, name):
         command = ["java", "-jar",


Mime
View raw message