From commits-return-102870-archive-asf-public=cust-asf.ponee.io@beam.apache.org Thu Aug 1 22:18:42 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 10D2B180644 for ; Fri, 2 Aug 2019 00:18:41 +0200 (CEST) Received: (qmail 41359 invoked by uid 500); 1 Aug 2019 22:18:41 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 41350 invoked by uid 99); 1 Aug 2019 22:18:41 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Aug 2019 22:18:41 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id 5C7BF816C5; Thu, 1 Aug 2019 22:18:41 +0000 (UTC) Date: Thu, 01 Aug 2019 22:18:39 +0000 To: "commits@beam.apache.org" Subject: [beam] branch master updated: [BEAM-6611] Move limitations comment to TriggerCopyJobs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <156469791742.23918.16616521868551209565@gitbox.apache.org> From: pabloem@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: beam X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 850e8469de798d45ec535fe90cb2dc5dbda4974a X-Git-Newrev: 7c51e13c19987039f2adb4c32984c3299c5554f0 X-Git-Rev: 7c51e13c19987039f2adb4c32984c3299c5554f0 X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. pabloem pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 7c51e13 [BEAM-6611] Move limitations comment to TriggerCopyJobs new 2129c31 Merge pull request #9216 from ttanay/bq-copy-comment 7c51e13 is described below commit 7c51e13c19987039f2adb4c32984c3299c5554f0 Author: ttanay AuthorDate: Thu Aug 1 23:26:41 2019 +0530 [BEAM-6611] Move limitations comment to TriggerCopyJobs --- sdks/python/apache_beam/io/gcp/bigquery.py | 5 ----- sdks/python/apache_beam/io/gcp/bigquery_file_loads.py | 4 ++++ 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/sdks/python/apache_beam/io/gcp/bigquery.py b/sdks/python/apache_beam/io/gcp/bigquery.py index facbf9c..4f0a762 100644 --- a/sdks/python/apache_beam/io/gcp/bigquery.py +++ b/sdks/python/apache_beam/io/gcp/bigquery.py @@ -895,11 +895,6 @@ class WriteToBigQuery(PTransform): This transform receives a PCollection of elements to be inserted into BigQuery tables. The elements would come in as Python dictionaries, or as `TableRow` instances. - - Known Limitations/Edge Cases: - 1. [BEAM-7822] In file loads method of writing to BigQuery, - copying from temp_tables to destination_table is not atomic. - See: https://issues.apache.org/jira/browse/BEAM-7822 """ class Method(object): diff --git a/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py b/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py index a6871cd..b5b7c19 100644 --- a/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py +++ b/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py @@ -268,6 +268,10 @@ class TriggerCopyJobs(beam.DoFn): destination tables. This transform emits (destination, job_reference) pairs. + + TODO(BEAM-7822): In file loads method of writing to BigQuery, + copying from temp_tables to destination_table is not atomic. + See: https://issues.apache.org/jira/browse/BEAM-7822 """ def __init__(self, create_disposition=None,