nemo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jooy...@apache.org
Subject [incubator-nemo] branch maven-artifact updated (49266b5 -> 85d2507)
Date Thu, 01 Nov 2018 02:03:15 GMT
This is an automated email from the ASF dual-hosted git repository.

jooykim pushed a change to branch maven-artifact
in repository https://gitbox.apache.org/repos/asf/incubator-nemo.git.


    from 49266b5  Merge branch 'master' into maven-artifact
     add 04d6eb4  [NEMO-230] Emit collected data when receiving watermark in GroupByKeyAndWindowTransform
(#135)
     add 53522c6  [NEMO-237] Refactor ParentTaskDataFetcher to emit streaming data and watermark
(#138)
     add 7858051  [NEMO-232] Implement InputWatermarkManager (#137)
     new 85d2507  Merge branch 'master' into maven-artifact

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../GroupByKeyAndWindowDoFnTransform.java          | 165 ++++++++++---
 .../frontend/beam/transform/DoFnTransformTest.java |  35 ---
 .../GroupByKeyAndWindowDoFnTransformTest.java      | 176 +++++++++++++
 .../beam/transform/TestOutputCollector.java        |  69 ++++++
 pom.xml                                            |  16 +-
 .../runtime/executor/data/PipeManagerWorker.java   |   8 +-
 .../datatransfer/DataFetcherOutputCollector.java   |   1 +
 .../datatransfer/InputWatermarkManager.java        |  46 ++++
 .../datatransfer/MultiInputWatermarkManager.java   |  90 +++++++
 .../datatransfer/NextIntraTaskOperatorInfo.java    |  59 +++++
 .../OperatorVertexOutputCollector.java             |  35 ++-
 ...eader.java => SingleInputWatermarkManager.java} |  35 +--
 .../task/MultiThreadParentTaskDataFetcher.java     | 150 +++++++++++
 .../executor/task/ParentTaskDataFetcher.java       |   1 -
 .../nemo/runtime/executor/task/TaskExecutor.java   | 111 +++++++--
 .../datatransfer/InputWatermarkManagerTest.java    |  88 +++++++
 .../runtime/executor/task/TaskExecutorTest.java    | 274 +++++++++++++++------
 17 files changed, 1145 insertions(+), 214 deletions(-)
 create mode 100644 compiler/test/src/test/java/org/apache/nemo/compiler/frontend/beam/transform/GroupByKeyAndWindowDoFnTransformTest.java
 create mode 100644 compiler/test/src/test/java/org/apache/nemo/compiler/frontend/beam/transform/TestOutputCollector.java
 create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/InputWatermarkManager.java
 create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/MultiInputWatermarkManager.java
 create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/NextIntraTaskOperatorInfo.java
 copy runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/{InputReader.java
=> SingleInputWatermarkManager.java} (53%)
 create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/task/MultiThreadParentTaskDataFetcher.java
 create mode 100644 runtime/executor/src/test/java/org/apache/nemo/runtime/executor/datatransfer/InputWatermarkManagerTest.java


Mime
View raw message