beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Problem with Pipeline In Flink Runner
Date Thu, 19 May 2016 16:52:17 GMT
Hi Ismaël,

There is a pull request which enables Windows on batch. It will be
merged soon. In the meantime, please use the streaming execution mode
of the Flink Runner.

Use setStreaming(true) to enable it on the FlinkPipelineOptions.

Cheers,
Max

On Thu, May 19, 2016 at 6:32 PM, Ismaël Mejía <iemejia@gmail.com> wrote:
> I was trying to run a local Pipeline we have (basically a batch one we
> rewrote from TPC-H)  in all the possible runners, for the moment I had
> success with the DirectPipelineRunner, InProcessPipelineRunner and
> SparkPipelineRunner. However when I executed it in the FlinkPipelineRunner I
> got this exception:
>
> java.lang.UnsupportedOperationException: The transform Window.Into() is
> currently not supported.
>
> I thought it was probably a configuration error since I remember that in the
> flink runner we had to explicitly define if it was a batch or streaming job,
> but when I checked the options:
>
> Current Settings:
>   appName: TestQ08Task
>   filesToStage: ...
>   flinkMaster: [auto]
>   parallelism: 1
>   runner: class org.apache.beam.runners.flink.FlinkPipelineRunner
>   stableUniqueNames: WARNING
>   streaming: false
>
> I noticed I was in batch mode, is there something missing in the options
> configuration, or is it that the UnsoppertedOperation is a WIP or not
> supported at all by Flink ?
>
> And extra question, this option to define if it is in batch or stream mode
> is going to stay for long, can't be this inferred some how ?
>
> Thanks,
> -Ismaël
>

Mime
View raw message