airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From criccom...@apache.org
Subject incubator-airflow git commit: [AIRFLOW-1718] Set num_retries on Dataproc job request execution
Date Wed, 18 Oct 2017 18:30:26 GMT
Repository: incubator-airflow
Updated Branches:
  refs/heads/master b7a1f8b9c -> 6078e753a


[AIRFLOW-1718] Set num_retries on Dataproc job request execution

Closes #2696 from cjqian/1718


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/6078e753
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/6078e753
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/6078e753

Branch: refs/heads/master
Commit: 6078e753aac35aa4f5971a719d9f736c35396770
Parents: b7a1f8b
Author: Crystal Qian <crystaljqian@gmail.com>
Authored: Wed Oct 18 11:30:07 2017 -0700
Committer: Chris Riccomini <criccomini@apache.org>
Committed: Wed Oct 18 11:30:11 2017 -0700

----------------------------------------------------------------------
 airflow/contrib/hooks/gcp_dataproc_hook.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/6078e753/airflow/contrib/hooks/gcp_dataproc_hook.py
----------------------------------------------------------------------
diff --git a/airflow/contrib/hooks/gcp_dataproc_hook.py b/airflow/contrib/hooks/gcp_dataproc_hook.py
index 4be166e..5b96484 100644
--- a/airflow/contrib/hooks/gcp_dataproc_hook.py
+++ b/airflow/contrib/hooks/gcp_dataproc_hook.py
@@ -41,7 +41,7 @@ class _DataProcJob(LoggingMixin):
             self.job = self.dataproc_api.projects().regions().jobs().get(
                 projectId=self.project_id,
                 region=self.region,
-                jobId=self.job_id).execute()
+                jobId=self.job_id).execute(num_retries=5)
             if 'ERROR' == self.job['status']['state']:
                 print(str(self.job))
                 self.log.error('DataProc job %s has errors', self.job_id)


Mime
View raw message