Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7EE8D200CF7 for ; Tue, 19 Sep 2017 19:19:28 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 7D5641609DD; Tue, 19 Sep 2017 17:19:28 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id BADB61609BF for ; Tue, 19 Sep 2017 19:19:27 +0200 (CEST) Received: (qmail 21706 invoked by uid 500); 19 Sep 2017 17:19:26 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 21697 invoked by uid 99); 19 Sep 2017 17:19:26 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Sep 2017 17:19:26 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id CD84AF3292; Tue, 19 Sep 2017 17:19:26 +0000 (UTC) From: reuvenlax To: commits@beam.apache.org Reply-To: commits@beam.apache.org Message-ID: Subject: [GitHub] beam pull request #3863: [Beam-2858] Ensure that file deletion happens consi... Content-Type: text/plain Date: Tue, 19 Sep 2017 17:19:26 +0000 (UTC) archived-at: Tue, 19 Sep 2017 17:19:28 -0000 GitHub user reuvenlax opened a pull request: https://github.com/apache/beam/pull/3863 [Beam-2858] Ensure that file deletion happens consistently, and only after table loads are completed We do this by adding a GroupByKey between the BQ load and the file deletion. Once the @StableReplay proposal is implemented, we will replace with a solution that uses that instead. R: @jkff You can merge this pull request into a Git repository by running: $ git pull https://github.com/reuvenlax/incubator-beam BEAM-2858 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3863.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3863 ---- commit c203331efdd24551e5f7270c55507791fdc9367b Author: Reuven Lax Date: 2017-09-19T17:10:18Z Fix bug. commit 6cfacb8a6324d9b5a69c7d1dae82e6c1581d85af Author: Reuven Lax Date: 2017-09-19T17:17:51Z Some fixups. ---- ---