beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (BEAM-6) Import Spark Runner code
Date Thu, 10 Mar 2016 11:30:41 GMT


ASF GitHub Bot commented on BEAM-6:

GitHub user tomwhite opened a pull request:

    Import Spark Runner code

    This addresses
    I've preserved git history (using @mxm's amazing git rewriting trick from #12). This is
just an initial import - the Spark runner build is not yet integrated with the main build,
packages need changing, etc. That's going to take more work, so it might be a good idea to
get this merged first.

You can merge this pull request into a Git repository by running:

    $ git pull beam-6-import-spark-runner

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #37
commit ad444470e97f7a1a9bc4582831f9d6df334e9949
Author: Josh Wills <>
Date:   2014-07-11T01:59:48Z

    Initial commit

commit 845a8178ab8bb620d0278b8312f04dce3f1348d8
Author: Josh Wills <>
Date:   2014-07-11T04:25:35Z

    Dumbest proof of concept possible

commit cb7c86642ea9839e3638c16849e113cb3536604a
Author: Josh Wills <>
Date:   2014-11-04T00:40:14Z

    First bit of work to get this running against the new Dataflow API

commit dce03e4b7d607bf8ac9846fa4bfb26a81d747a63
Author: Josh Wills <>
Date:   2014-12-02T01:58:30Z

    Update version of dataflow to get new API method access

commit 08e94b28ea06667049817def8aa5aee7aa31ebe1
Author: Josh Wills <>
Date:   2014-12-02T02:23:05Z

    Add support for getters and a Flatten impl

commit 9fdac6ccb750488cdff8e3a696d114911d385ee4
Author: Josh Wills <>
Date:   2014-12-02T06:54:28Z

    Such code. Much features.

commit deca2c0bdbe699099717591fc89afa4175c5438c
Author: Josh Wills <>
Date:   2014-12-02T23:28:15Z

    Adding some more operators: toiterable, seqdo

commit 6ee38b2a0677fc8eb77b4cd0aca64cfec2ec7fd9
Author: Josh Wills <>
Date:   2014-12-03T00:31:31Z

    Support for ParDo.BoundMulti

commit 64c6d8d723023aafc2ee5754d41e00eca1524ddf
Author: Josh Wills <>
Date:   2014-12-03T00:54:14Z

    Fix bug in deserializing side inputs

commit 40adbeceb32b5d8a2a01bb5156c4e97ae9cfc58c
Author: Josh Wills <>
Date:   2014-12-03T03:13:24Z

    Add SparkRuntimeContext for handling shared runtime objects

commit 565509dc50a0f216665e6900a65842509b63e4d5
Author: Josh Wills <>
Date:   2014-12-03T04:12:49Z

    First cut at aggregators

commit bb219d48a89f4f61d090b7cf47566dcee85adcf9
Author: Josh Wills <>
Date:   2014-12-03T05:03:35Z

    First minimally working aggregators

commit 9152769c79c00aa8c9332c23ffcdd42a9876f402
Author: Josh Wills <>
Date:   2014-12-09T06:40:34Z

    Updates for 141206 SDK release

commit 3bd04ae9ebcd89c93e5e635d806b613c5c5438ae
Author: Josh Wills <>
Date:   2014-12-09T20:06:28Z

    Dummy impls of windowing-related ProcContext functions

commit ba74f19ba6bda500a9541bbd84655336f1cff149
Author: Juliet Hougland <>
Date:   2014-12-09T20:16:13Z

    Simplify pom.xml

commit 6aa08e067d87ddb896df5a1a828bc0c526b9964f
Author: Josh Wills <>
Date:   2014-12-09T23:30:29Z

    Add proper coder handling to RDD retrieval

commit 45be5086820ba71dbee81bd7785f81e38f899fe8
Author: Juliet Hougland <>
Date:   2014-12-10T00:35:25Z

    Refactor aggregation related classes.

commit 137d54a1efe4e134cc7645c01eb013db10d2be30
Author: Juliet Hougland <>
Date:   2014-12-10T00:37:46Z

    Adds Javadoc and Tests to project.

commit 67cf364252fcf1df66ad97dac12d0cc006aa5d46
Author: Juliet Hougland <>
Date:   2014-12-11T20:12:45Z

    Add and update project version in pom.xml.

commit b9545895738856693f3b5a40e127e348763a91f8
Author: Juliet Hougland <>
Date:   2014-12-11T22:28:00Z

    Add apache2 license and cloudera copyright.

commit 1523ffdae2a2fdf8029dfab68da1430edc6fe951
Author: Juliet Hougland <>
Date:   2014-12-12T00:40:28Z

    Adds custom checkstyle.
    more checkstyle improvements

commit 2992838fe35e06f280fbbf2eb419973be7699281
Author: Juliet Hougland <>
Date:   2014-12-12T00:56:01Z

    Factor out tranform translation logic in to its own class.

commit 2e3fe1ab9412be1b10659cd273a4de0ec5f103cd
Author: Sean Owen <>
Date:   2014-12-13T16:03:39Z

    Specify and rationalize generic types in State, CoderHelpers to start

commit 74892633fa1c5a66f2f05cbce4b0e571ed72d71f
Author: Juliet Hougland <>
Date:   2014-12-16T21:15:05Z

    Add simple word count test.

commit f9e8fab69223582ab5b68cdfb634fad4c0c525ef
Author: Juliet Hougland <>
Date:   2014-12-17T01:34:57Z

    Factor out spark pipeline options.

commit ec172bac7b77774f86f0982d0261e1b7bb6d52e6
Author: Sean Owen <>
Date:   2014-12-19T21:37:23Z

    Miscellaneous inspection changes from IntelliJ

commit ed1e2f7fb66f0746c1a8511b398f41a6df0e1513
Author: Sean Owen <>
Date:   2014-12-20T08:36:13Z

    Issue #13 : attempt to remove all generics warnings, or handle them explicitly

commit 225f6c0121e6171b02097cac86724920e0e32b50
Author: Sean Owen <>
Date:   2014-12-21T18:31:44Z

    Update and specify POM plugin config; Update Spark to 1.1.1, JUnit to 4.12, Spark 1.2;
Add source, javadoc plugins and other info; Fix javadoc errors and a few typos

commit 1f9cd04591497231a65c83d6d3f4a27428be79ac
Author: Juliet Hougland <>
Date:   2014-12-24T02:55:18Z

    Improve readme to explain current state of the repo, and to encourage outside contributions.

commit ba4b32687c1c1bbf86bec6fc3a87e9c90c078ad7
Author: Juliet Hougland <>
Date:   2014-12-23T18:44:46Z

    Update version of dataflow we depend on.
    The primary change needed to accomodate the new dataflow api is to how we
    handle side inputs.


> Import Spark Runner code
> ------------------------
>                 Key: BEAM-6
>                 URL:
>             Project: Beam
>          Issue Type: Sub-task
>          Components: runner-spark
>            Reporter: Frances Perry
>            Assignee: Tom White

This message was sent by Atlassian JIRA

View raw message