Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id EE68F200C6A for ; Wed, 19 Apr 2017 19:56:46 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id ED044160BAD; Wed, 19 Apr 2017 17:56:46 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 479EB160B94 for ; Wed, 19 Apr 2017 19:56:46 +0200 (CEST) Received: (qmail 49130 invoked by uid 500); 19 Apr 2017 17:56:43 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 49121 invoked by uid 99); 19 Apr 2017 17:56:43 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Apr 2017 17:56:43 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 4C1D6C0040 for ; Wed, 19 Apr 2017 17:56:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 4x7KvPcXdA4R for ; Wed, 19 Apr 2017 17:56:42 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 812885FBBB for ; Wed, 19 Apr 2017 17:56:42 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 18759E00A6 for ; Wed, 19 Apr 2017 17:56:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id C8F9A20D59 for ; Wed, 19 Apr 2017 17:56:41 +0000 (UTC) Date: Wed, 19 Apr 2017 17:56:41 +0000 (UTC) From: "Daniel Halperin (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-2007) DataflowRunner drops Reads with no consumers MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 19 Apr 2017 17:56:47 -0000 [ https://issues.apache.org/jira/browse/BEAM-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15975169#comment-15975169 ] Daniel Halperin commented on BEAM-2007: --------------------------------------- [~altay] -- assigning to you solely as FYI > DataflowRunner drops Reads with no consumers > -------------------------------------------- > > Key: BEAM-2007 > URL: https://issues.apache.org/jira/browse/BEAM-2007 > Project: Beam > Issue Type: Bug > Components: runner-dataflow, sdk-py > Reporter: Daniel Halperin > Assignee: Ahmet Altay > Fix For: First stable release > > > Basically, if a pipeline has "just" a Read with no consumers, the optimizer in Dataflow will drop it. To preserve Beam semantics, we do want to run the Read and drop its output, e.g., because the Read may have side effects that we're testing for. > Is it possible with pipeline surgery to find such Reads and add an Identity ParDo to them? -- This message was sent by Atlassian JIRA (v6.3.15#6346)