Return-Path: X-Original-To: apmail-beam-commits-archive@minotaur.apache.org Delivered-To: apmail-beam-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 718B3184D1 for ; Thu, 28 Apr 2016 19:51:18 +0000 (UTC) Received: (qmail 54680 invoked by uid 500); 28 Apr 2016 19:51:18 -0000 Delivered-To: apmail-beam-commits-archive@beam.apache.org Received: (qmail 54629 invoked by uid 500); 28 Apr 2016 19:51:18 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 54620 invoked by uid 99); 28 Apr 2016 19:51:18 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2016 19:51:18 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id D1D721A21E8 for ; Thu, 28 Apr 2016 19:51:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.021 X-Spam-Level: X-Spam-Status: No, score=-4.021 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id Ug1SfJzNZdKX for ; Thu, 28 Apr 2016 19:51:15 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id CE8375F369 for ; Thu, 28 Apr 2016 19:51:14 +0000 (UTC) Received: (qmail 52938 invoked by uid 99); 28 Apr 2016 19:51:13 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2016 19:51:13 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 092D32C1F60 for ; Thu, 28 Apr 2016 19:51:13 +0000 (UTC) Date: Thu, 28 Apr 2016 19:51:13 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-22) DirectPipelineRunner: support for unbounded collections MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/BEAM-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15262824#comment-15262824 ] ASF GitHub Bot commented on BEAM-22: ------------------------------------ GitHub user tgroh opened a pull request: https://github.com/apache/incubator-beam/pull/260 [BEAM-22] Add CommittedResult Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [x] Replace `` in the title with the actual Jira issue number, if there is one. - [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Return as the output to InProcessEvaluationContext#handleResult(). This allows a richer return type to improve possible behaviors when a result is returned. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tgroh/incubator-beam ippr_committed_result_as_handle_result Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/260.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #260 ---- commit 13da09d9f1b4a6498cfde8067d31e96f5817ca74 Author: Thomas Groh Date: 2016-04-28T19:22:47Z Add CommittedResult Return as the output to InProcessEvaluationContext#handleResult(). This allows a richer return type to improve possible behaviors when a result is returned. ---- > DirectPipelineRunner: support for unbounded collections > ------------------------------------------------------- > > Key: BEAM-22 > URL: https://issues.apache.org/jira/browse/BEAM-22 > Project: Beam > Issue Type: Improvement > Components: runner-direct > Reporter: Davor Bonaci > Assignee: Thomas Groh > > DirectPipelineRunner currently runs over bounded PCollections only, and implements only a portion of the Beam Model. > We should improve it to faithfully implement the full Beam Model, such as add ability to run over unbounded PCollections, and better resemble execution model in a distributed system. > This further enables features such as a testing source which may simulate late data and test triggers in the pipeline. Finally, we may want to expose an option to select between "debug" (single threaded), "chaos monkey" (test as many model requirements as possible), and "performance" (multi-threaded). > more testing (chaos monkey) > Once this is done, we should update this StackOverflow question: > http://stackoverflow.com/questions/35350113/testing-triggers-with-processing-time/35401426#35401426 -- This message was sent by Atlassian JIRA (v6.3.4#6332)