beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henning Korsholm Rohde (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-186) The streaming WindowedWordCount example is broken on Dataflow
Date Sat, 09 Apr 2016 00:04:25 GMT
Henning Korsholm Rohde created BEAM-186:
-------------------------------------------

             Summary: The streaming WindowedWordCount example is broken on Dataflow
                 Key: BEAM-186
                 URL: https://issues.apache.org/jira/browse/BEAM-186
             Project: Beam
          Issue Type: Bug
          Components: examples-java
            Reporter: Henning Korsholm Rohde
            Assignee: Frances Perry
            Priority: Minor


The streaming version of WindowedWordCount example is broken on Dataflow. Currently, the injector
cannot start up and emits the error "java.lang.RuntimeException: DataflowWorkerHarness should
not be used for non-streaming pipelines". This in turn means that no messages are generated
for the streaming pipeline to process.

The problem is that DataflowExampleUtils uses a batch workflow to inject Google PubSub messages.
This workflow reuses the real options and modifies them to be batch, setting isStreaming to
false etc. However, with versioned containers, the option for workerHarnessContainerImage
must now also be changed to use the batch image.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message