Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 936C2200BD5 for ; Thu, 8 Dec 2016 20:19:06 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 91F8A160B1F; Thu, 8 Dec 2016 19:19:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DCBA4160B0A for ; Thu, 8 Dec 2016 20:19:05 +0100 (CET) Received: (qmail 96532 invoked by uid 500); 8 Dec 2016 19:19:05 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 96517 invoked by uid 99); 8 Dec 2016 19:19:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Dec 2016 19:19:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 86CB01A0943 for ; Thu, 8 Dec 2016 19:19:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.218 X-Spam-Level: X-Spam-Status: No, score=-6.218 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id s3BdVU0-RhZg for ; Thu, 8 Dec 2016 19:19:02 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id A3F9C5FAF3 for ; Thu, 8 Dec 2016 19:19:01 +0000 (UTC) Received: (qmail 96356 invoked by uid 99); 8 Dec 2016 19:19:00 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Dec 2016 19:19:00 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 88E49E36DC; Thu, 8 Dec 2016 19:19:00 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: robertwb@apache.org To: commits@beam.incubator.apache.org Date: Thu, 08 Dec 2016 19:19:00 -0000 Message-Id: <24aaf1941da042e39c4b9b8f0ba9d36b@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/2] incubator-beam git commit: Handle empty batches in GcsIO batch methods archived-at: Thu, 08 Dec 2016 19:19:06 -0000 Repository: incubator-beam Updated Branches: refs/heads/python-sdk 75be6e974 -> 1392f70b6 Handle empty batches in GcsIO batch methods Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3ef83b33 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3ef83b33 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/3ef83b33 Branch: refs/heads/python-sdk Commit: 3ef83b3396a4574b3283b29ba1f878b31badd612 Parents: 75be6e9 Author: Charles Chen Authored: Wed Dec 7 15:03:01 2016 -0800 Committer: Robert Bradshaw Committed: Thu Dec 8 11:18:51 2016 -0800 ---------------------------------------------------------------------- sdks/python/apache_beam/io/gcsio.py | 4 ++++ sdks/python/apache_beam/io/gcsio_test.py | 4 ++++ 2 files changed, 8 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/3ef83b33/sdks/python/apache_beam/io/gcsio.py ---------------------------------------------------------------------- diff --git a/sdks/python/apache_beam/io/gcsio.py b/sdks/python/apache_beam/io/gcsio.py index 748465f..f150c4c 100644 --- a/sdks/python/apache_beam/io/gcsio.py +++ b/sdks/python/apache_beam/io/gcsio.py @@ -204,6 +204,8 @@ class GcsIO(object): argument, where exception is None if the operation succeeded or the relevant exception if the operation failed. """ + if not paths: + return [] batch_request = BatchApiRequest( retryable_codes=retry.SERVER_ERROR_OR_TIMEOUT_CODES) for path in paths: @@ -264,6 +266,8 @@ class GcsIO(object): src_dest_pairs argument, where exception is None if the operation succeeded or the relevant exception if the operation failed. """ + if not src_dest_pairs: + return [] batch_request = BatchApiRequest( retryable_codes=retry.SERVER_ERROR_OR_TIMEOUT_CODES) for src, dest in src_dest_pairs: http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/3ef83b33/sdks/python/apache_beam/io/gcsio_test.py ---------------------------------------------------------------------- diff --git a/sdks/python/apache_beam/io/gcsio_test.py b/sdks/python/apache_beam/io/gcsio_test.py index 5af13c6..bd7eb51 100644 --- a/sdks/python/apache_beam/io/gcsio_test.py +++ b/sdks/python/apache_beam/io/gcsio_test.py @@ -265,6 +265,10 @@ class TestGCSIO(unittest.TestCase): with self.assertRaises(ValueError): self.gcs.open(file_name, 'r+b') + def test_empty_batches(self): + self.assertEqual([], self.gcs.copy_batch([])) + self.assertEqual([], self.gcs.delete_batch([])) + def test_delete(self): file_name = 'gs://gcsio-test/delete_me' file_size = 1024