beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From k...@apache.org
Subject [50/50] [abbrv] beam git commit: This closes #3636: Merge GearpumpRunner to master branch
Date Mon, 07 Aug 2017 23:54:15 GMT
This closes #3636: Merge GearpumpRunner to master branch

  Remove gearpump-runner in precommit tests
  Override maven-enforcer-plugin for precommit tests
  Require java 8 for precommit tests and add gearpump-runner
  Deactivate integration-tests for gearpump-runner by default
  Add beam-runners-gearpump dependency to javadoc
  Upgrade BEAM version to 2.2.0-SNAPSHOT in gearpump-runner
  Revert accidental changes to sdks/java/pom.xml
  Fix ParDoTest#testPipelineOptionsParameter
  Upgrade to gearpump 0.8.4
  Remove unused codes
  Fix kryo exception
  Fix PCollectionView translation
  upgrade to gearpump 0.8.4-SNAPSHOT
  Fix side input handling in DoFnFunction
  Respect WindowFn#getOutputTime in gearpump-runner
  Activate Gearpump local-validates-runner-tests in precommit
  Update against master changes
  Update gearpump-runner against master changes
  Update gearpump-runner against master changes
  Update gearpump-runner against master changes.
  [BEAM-972] Add more unit test to Gearpump runner
  [BEAM-972] Add unit tests to Gearpump runner
  [BEAM-79] Fix gearpump-runner merge conflicts and test failure
  enable ParDoTest
  [BEAM-79] Add SideInput support for GearpumpRunner
  [BEAM-79] Support merging windows in GearpumpRunner
  [BEAM-79] Fix PostCommit test confs for Gearpump runner
  note thread is interrupted on InterruptedException
  Remove cache for Gearpump on travis
  reduce timeout to wait for result
  fix ParDo.BoundMulti translation
  return encoded key for GroupByKey translation
  support OutputTimeFn
  update to latest gearpump dsl function interface
  fix group by window
  activate ROS on Gearpump by default
  update ROS configurations
  [BEAM-1180] Implement GearpumpPipelineResult
  [BEAM-79] Upgrade to beam-0.5.0-incubating-SNAPSHOT
  [BEAM-79] Update to latest Gearpump API
  Fix NoOpAggregatorFactory
  Remove print to stdout
  Skip window assignment when windows don't change
  Add Window.Bound translator
  Upgrade Gearpump version
  [BEAM-79] fix gearpump runner build failure
  [BEAM-79] update GearpumpPipelineResult
  [BEAM-79] Port Gearpump runner from OldDoFn to new DoFn
  upgrade gearpump-runner to 0.4.0-incubating-SNAPSHOT
  remove "pipeline" in runner name
  post-merge fix
  [BEAM-79] fix integration-test failure
  fix import order
  !fixup Minor javadoc clean-up
  Added even more javadoc to TextIO#withHeader and TextIO#withFooter (2).
  Added even more javadoc to TextIO#withHeader and TextIO#withFooter.
  Added javadoc to TextIO#withHeader and TextIO#withFooter.
  Reverted header and footer to be of type String.
  Revised according to comments following a code review.
  Add header/footer support to TextIO.Write
  [BEAM-242] Enable and fix checkstyle in Flink runner examples
  Remove timeout in JAXBCoderTest
  Be more accepting in UnboundedReadDeduplicatorTest
  BigQuery: limit max job polling time to 1 minute
  [BEAM-242] Enable checkstyle and fix checkstyle errors in Flink runner
  [BEAM-456] Add MongoDbIO
  FluentBackoff: a replacement for a variety of custom backoff implementations
  Remove the DataflowRunner instructions from examples
  Put classes in runners-core package into runners.core namespace
  Delegate populateDipslayData to wrapped combineFn's
  Fixed Combine display data
  Cloud Datastore naming clean-up
  DatastoreIO SplitQueryFn integration test
  Add Latest CombineFn and PTransforms
  Remove empty unused method in TestStreamEvaluatorFactory
  Test that multiple instances of TestStream are supported
  Correct some accidental renames
  Fix condition in FlinkStreamingPipelineTranslator
  Address comments of Flink Side-Input PR
  [BEAM-569] Define maxNumRecords default value to Long.MAX_VALUE in JmsIO
  Add LeaderBoardTest
  take advantage of setup/teardown for KafkaWriter
  Returned KafkaIO getWatermark log line in debug mode
  [BEAM-572] Remove Spark Reference in WordCount
  Update Dataflow Container Version
  [BEAM-313] Provide a context for SparkRunner
  DataflowRunner: get PBegin from PInput
  [BEAM-592] Fix SparkRunner Dependency Problem in WordCount
  Fix javadoc in Kinesis
  Organize imports in Kinesis
  kinesis: a connector for Amazon Kinesis
  [BEAM-589] Fixing IO.Read transformation
  Query latest timestamp
  travis.yml: disable updating snapshots
  Added support for reporting aggregator values to Spark sinks
  [BEAM-294] Rename dataflow references to beam
  Modified BigtableIO to use DoFn setup/tearDown methods instead of startBundle/finishBundle
  checkstyle: prohibit API client repackaged Guava
  Make WriteTest more resilient to Randomness
  Update DoFn javadocs to remove references to OldDoFn and Dataflow
  ...


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/89236e3b
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/89236e3b
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/89236e3b

Branch: refs/heads/master
Commit: 89236e3b588cd8ab5092667b1fe155738c69dbcf
Parents: c9abd15 c35febf
Author: Kenneth Knowles <klk@google.com>
Authored: Mon Aug 7 16:51:42 2017 -0700
Committer: Kenneth Knowles <klk@google.com>
Committed: Mon Aug 7 16:51:42 2017 -0700

----------------------------------------------------------------------
 ...tCommit_Java_ValidatesRunner_Gearpump.groovy |   2 +-
 examples/java8/pom.xml                          |  12 +
 pom.xml                                         |   6 +
 runners/gearpump/README.md                      |  61 ++++
 runners/gearpump/pom.xml                        | 280 +++++++++++++++++++
 .../gearpump/GearpumpPipelineOptions.java       |  67 +++++
 .../gearpump/GearpumpPipelineResult.java        | 109 ++++++++
 .../beam/runners/gearpump/GearpumpRunner.java   | 119 ++++++++
 .../gearpump/GearpumpRunnerRegistrar.java       |  64 +++++
 .../runners/gearpump/TestGearpumpRunner.java    |  63 +++++
 .../beam/runners/gearpump/package-info.java     |  22 ++
 ...CreateGearpumpPCollectionViewTranslator.java |  45 +++
 .../CreateStreamingGearpumpView.java            | 157 +++++++++++
 .../FlattenPCollectionsTranslator.java          |  83 ++++++
 .../translators/GearpumpPipelineTranslator.java | 143 ++++++++++
 .../translators/GroupByKeyTranslator.java       | 258 +++++++++++++++++
 .../translators/ParDoMultiOutputTranslator.java | 103 +++++++
 .../translators/ReadBoundedTranslator.java      |  46 +++
 .../translators/ReadUnboundedTranslator.java    |  48 ++++
 .../translators/TransformTranslator.java        |  30 ++
 .../translators/TranslationContext.java         | 105 +++++++
 .../translators/WindowAssignTranslator.java     | 103 +++++++
 .../translators/functions/DoFnFunction.java     | 193 +++++++++++++
 .../translators/functions/package-info.java     |  22 ++
 .../translators/io/BoundedSourceWrapper.java    |  45 +++
 .../gearpump/translators/io/GearpumpSource.java | 113 ++++++++
 .../translators/io/UnboundedSourceWrapper.java  |  46 +++
 .../gearpump/translators/io/ValuesSource.java   | 173 ++++++++++++
 .../gearpump/translators/io/package-info.java   |  22 ++
 .../gearpump/translators/package-info.java      |  22 ++
 .../translators/utils/DoFnRunnerFactory.java    |  82 ++++++
 .../translators/utils/NoOpStepContext.java      |  41 +++
 .../translators/utils/TranslatorUtils.java      | 218 +++++++++++++++
 .../translators/utils/package-info.java         |  22 ++
 .../gearpump/GearpumpRunnerRegistrarTest.java   |  55 ++++
 .../runners/gearpump/PipelineOptionsTest.java   |  73 +++++
 ...teGearpumpPCollectionViewTranslatorTest.java |  56 ++++
 .../FlattenPCollectionsTranslatorTest.java      | 149 ++++++++++
 .../translators/GroupByKeyTranslatorTest.java   | 152 ++++++++++
 .../translators/ReadBoundedTranslatorTest.java  |  70 +++++
 .../ReadUnboundedTranslatorTest.java            |  70 +++++
 .../translators/WindowAssignTranslatorTest.java | 110 ++++++++
 .../translators/io/GearpumpSourceTest.java      |  99 +++++++
 .../gearpump/translators/io/ValueSoureTest.java |  79 ++++++
 .../translators/utils/TranslatorUtilsTest.java  |  73 +++++
 runners/pom.xml                                 |   9 +
 sdks/java/javadoc/pom.xml                       |   5 +
 47 files changed, 3924 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/beam/blob/89236e3b/examples/java8/pom.xml
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/beam/blob/89236e3b/pom.xml
----------------------------------------------------------------------


Mime
View raw message