airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [airflow] nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery operators idempotency
Date Wed, 30 Oct 2019 16:56:38 GMT
nuclearpinguin commented on a change in pull request #6470: [AIRFLOW-5817] Improve BigQuery
operators idempotency
URL: https://github.com/apache/airflow/pull/6470#discussion_r340740308
 
 

 ##########
 File path: airflow/gcp/hooks/bigquery.py
 ##########
 @@ -332,19 +332,13 @@ def create_empty_table(self,
         self.log.info('Creating Table %s:%s.%s',
                       project_id, dataset_id, table_id)
 
-        try:
-            self.service.tables().insert(
-                projectId=project_id,
-                datasetId=dataset_id,
-                body=table_resource).execute(num_retries=num_retries)
-
-            self.log.info('Table created successfully: %s:%s.%s',
-                          project_id, dataset_id, table_id)
+        self.service.tables().insert(
+            projectId=project_id,
+            datasetId=dataset_id,
+            body=table_resource).execute(num_retries=num_retries)
 
-        except HttpError as err:
-            raise AirflowException(
-                'BigQuery job failed. Error was: {}'.format(err.content)
-            )
+        self.log.info('Table created successfully: %s:%s.%s',
 
 Review comment:
   In operator in case of http error 409 we assume the resource already exists. Previously
an error was thrown like "Database already exists", thus the operator was not idempotent.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message