Return-Path: X-Original-To: apmail-beam-commits-archive@minotaur.apache.org Delivered-To: apmail-beam-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A93FC1923F for ; Thu, 10 Mar 2016 11:30:45 +0000 (UTC) Received: (qmail 90836 invoked by uid 500); 10 Mar 2016 11:30:45 -0000 Delivered-To: apmail-beam-commits-archive@beam.apache.org Received: (qmail 90783 invoked by uid 500); 10 Mar 2016 11:30:45 -0000 Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.incubator.apache.org Delivered-To: mailing list commits@beam.incubator.apache.org Received: (qmail 90774 invoked by uid 99); 10 Mar 2016 11:30:45 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Mar 2016 11:30:45 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 202F5C2320 for ; Thu, 10 Mar 2016 11:30:45 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -3.771 X-Spam-Level: X-Spam-Status: No, score=-3.771 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, KAM_LOTSOFHASH=0.25, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id rdZrHmbeMUBX for ; Thu, 10 Mar 2016 11:30:44 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id B22D45F640 for ; Thu, 10 Mar 2016 11:30:42 +0000 (UTC) Received: (qmail 90664 invoked by uid 99); 10 Mar 2016 11:30:41 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Mar 2016 11:30:41 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 665232C1F58 for ; Thu, 10 Mar 2016 11:30:41 +0000 (UTC) Date: Thu, 10 Mar 2016 11:30:41 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-6) Import Spark Runner code MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/BEAM-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189165#comment-15189165 ] ASF GitHub Bot commented on BEAM-6: ----------------------------------- GitHub user tomwhite opened a pull request: https://github.com/apache/incubator-beam/pull/37 Import Spark Runner code This addresses https://issues.apache.org/jira/browse/BEAM-6. I've preserved git history (using @mxm's amazing git rewriting trick from #12). This is just an initial import - the Spark runner build is not yet integrated with the main build, packages need changing, etc. That's going to take more work, so it might be a good idea to get this merged first. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tomwhite/incubator-beam beam-6-import-spark-runner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/37.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #37 ---- commit ad444470e97f7a1a9bc4582831f9d6df334e9949 Author: Josh Wills Date: 2014-07-11T01:59:48Z Initial commit commit 845a8178ab8bb620d0278b8312f04dce3f1348d8 Author: Josh Wills Date: 2014-07-11T04:25:35Z Dumbest proof of concept possible commit cb7c86642ea9839e3638c16849e113cb3536604a Author: Josh Wills Date: 2014-11-04T00:40:14Z First bit of work to get this running against the new Dataflow API commit dce03e4b7d607bf8ac9846fa4bfb26a81d747a63 Author: Josh Wills Date: 2014-12-02T01:58:30Z Update version of dataflow to get new API method access commit 08e94b28ea06667049817def8aa5aee7aa31ebe1 Author: Josh Wills Date: 2014-12-02T02:23:05Z Add support for getters and a Flatten impl commit 9fdac6ccb750488cdff8e3a696d114911d385ee4 Author: Josh Wills Date: 2014-12-02T06:54:28Z Such code. Much features. commit deca2c0bdbe699099717591fc89afa4175c5438c Author: Josh Wills Date: 2014-12-02T23:28:15Z Adding some more operators: toiterable, seqdo commit 6ee38b2a0677fc8eb77b4cd0aca64cfec2ec7fd9 Author: Josh Wills Date: 2014-12-03T00:31:31Z Support for ParDo.BoundMulti commit 64c6d8d723023aafc2ee5754d41e00eca1524ddf Author: Josh Wills Date: 2014-12-03T00:54:14Z Fix bug in deserializing side inputs commit 40adbeceb32b5d8a2a01bb5156c4e97ae9cfc58c Author: Josh Wills Date: 2014-12-03T03:13:24Z Add SparkRuntimeContext for handling shared runtime objects commit 565509dc50a0f216665e6900a65842509b63e4d5 Author: Josh Wills Date: 2014-12-03T04:12:49Z First cut at aggregators commit bb219d48a89f4f61d090b7cf47566dcee85adcf9 Author: Josh Wills Date: 2014-12-03T05:03:35Z First minimally working aggregators commit 9152769c79c00aa8c9332c23ffcdd42a9876f402 Author: Josh Wills Date: 2014-12-09T06:40:34Z Updates for 141206 SDK release commit 3bd04ae9ebcd89c93e5e635d806b613c5c5438ae Author: Josh Wills Date: 2014-12-09T20:06:28Z Dummy impls of windowing-related ProcContext functions commit ba74f19ba6bda500a9541bbd84655336f1cff149 Author: Juliet Hougland Date: 2014-12-09T20:16:13Z Simplify pom.xml commit 6aa08e067d87ddb896df5a1a828bc0c526b9964f Author: Josh Wills Date: 2014-12-09T23:30:29Z Add proper coder handling to RDD retrieval commit 45be5086820ba71dbee81bd7785f81e38f899fe8 Author: Juliet Hougland Date: 2014-12-10T00:35:25Z Refactor aggregation related classes. commit 137d54a1efe4e134cc7645c01eb013db10d2be30 Author: Juliet Hougland Date: 2014-12-10T00:37:46Z Adds Javadoc and Tests to project. commit 67cf364252fcf1df66ad97dac12d0cc006aa5d46 Author: Juliet Hougland Date: 2014-12-11T20:12:45Z Add README.md and update project version in pom.xml. commit b9545895738856693f3b5a40e127e348763a91f8 Author: Juliet Hougland Date: 2014-12-11T22:28:00Z Add apache2 license and cloudera copyright. commit 1523ffdae2a2fdf8029dfab68da1430edc6fe951 Author: Juliet Hougland Date: 2014-12-12T00:40:28Z Adds custom checkstyle. more checkstyle improvements commit 2992838fe35e06f280fbbf2eb419973be7699281 Author: Juliet Hougland Date: 2014-12-12T00:56:01Z Factor out tranform translation logic in to its own class. commit 2e3fe1ab9412be1b10659cd273a4de0ec5f103cd Author: Sean Owen Date: 2014-12-13T16:03:39Z Specify and rationalize generic types in State, CoderHelpers to start commit 74892633fa1c5a66f2f05cbce4b0e571ed72d71f Author: Juliet Hougland Date: 2014-12-16T21:15:05Z Add simple word count test. commit f9e8fab69223582ab5b68cdfb634fad4c0c525ef Author: Juliet Hougland Date: 2014-12-17T01:34:57Z Factor out spark pipeline options. commit ec172bac7b77774f86f0982d0261e1b7bb6d52e6 Author: Sean Owen Date: 2014-12-19T21:37:23Z Miscellaneous inspection changes from IntelliJ commit ed1e2f7fb66f0746c1a8511b398f41a6df0e1513 Author: Sean Owen Date: 2014-12-20T08:36:13Z Issue #13 : attempt to remove all generics warnings, or handle them explicitly commit 225f6c0121e6171b02097cac86724920e0e32b50 Author: Sean Owen Date: 2014-12-21T18:31:44Z Update and specify POM plugin config; Update Spark to 1.1.1, JUnit to 4.12, Spark 1.2; Add source, javadoc plugins and other info; Fix javadoc errors and a few typos commit 1f9cd04591497231a65c83d6d3f4a27428be79ac Author: Juliet Hougland Date: 2014-12-24T02:55:18Z Improve readme to explain current state of the repo, and to encourage outside contributions. commit ba4b32687c1c1bbf86bec6fc3a87e9c90c078ad7 Author: Juliet Hougland Date: 2014-12-23T18:44:46Z Update version of dataflow we depend on. The primary change needed to accomodate the new dataflow api is to how we handle side inputs. ---- > Import Spark Runner code > ------------------------ > > Key: BEAM-6 > URL: https://issues.apache.org/jira/browse/BEAM-6 > Project: Beam > Issue Type: Sub-task > Components: runner-spark > Reporter: Frances Perry > Assignee: Tom White > -- This message was sent by Atlassian JIRA (v6.3.4#6332)