Return-Path: X-Original-To: apmail-beam-commits-archive@minotaur.apache.org Delivered-To: apmail-beam-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E0065199A1 for ; Fri, 29 Apr 2016 18:02:21 +0000 (UTC) Received: (qmail 93934 invoked by uid 500); 29 Apr 2016 18:02:21 -0000 Delivered-To: apmail-beam-commits-archive@beam.apache.org Received: (qmail 93884 invoked by uid 500); 29 Apr 2016 18:02:21 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 93875 invoked by uid 99); 29 Apr 2016 18:02:21 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Apr 2016 18:02:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 6D29CC0227 for ; Fri, 29 Apr 2016 18:02:21 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.021 X-Spam-Level: X-Spam-Status: No, score=-4.021 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id hm6x5lMDQ8J2 for ; Fri, 29 Apr 2016 18:02:19 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with SMTP id 16C3E5F640 for ; Fri, 29 Apr 2016 18:02:18 +0000 (UTC) Received: (qmail 89849 invoked by uid 99); 29 Apr 2016 18:02:13 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Apr 2016 18:02:13 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id ED1162C1F64 for ; Fri, 29 Apr 2016 18:02:12 +0000 (UTC) Date: Fri, 29 Apr 2016 18:02:12 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-22) DirectPipelineRunner: support for unbounded collections MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/BEAM-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264449#comment-15264449 ] ASF GitHub Bot commented on BEAM-22: ------------------------------------ Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/257 > DirectPipelineRunner: support for unbounded collections > ------------------------------------------------------- > > Key: BEAM-22 > URL: https://issues.apache.org/jira/browse/BEAM-22 > Project: Beam > Issue Type: Improvement > Components: runner-direct > Reporter: Davor Bonaci > Assignee: Thomas Groh > > DirectPipelineRunner currently runs over bounded PCollections only, and implements only a portion of the Beam Model. > We should improve it to faithfully implement the full Beam Model, such as add ability to run over unbounded PCollections, and better resemble execution model in a distributed system. > This further enables features such as a testing source which may simulate late data and test triggers in the pipeline. Finally, we may want to expose an option to select between "debug" (single threaded), "chaos monkey" (test as many model requirements as possible), and "performance" (multi-threaded). > more testing (chaos monkey) > Once this is done, we should update this StackOverflow question: > http://stackoverflow.com/questions/35350113/testing-triggers-with-processing-time/35401426#35401426 -- This message was sent by Atlassian JIRA (v6.3.4#6332)