beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-2806) support View.CreatePCollectionView in FlinkRunner
Date Fri, 25 Aug 2017 08:49:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16141378#comment-16141378
] 

Aljoscha Krettek commented on BEAM-2806:
----------------------------------------

Could you please provide a complete minimal example that provokes the problem. Normally, the
Flink Runner should support side input views, for example, the tests in {{ViewTest}} are executed
for the Flink Runner as well.

> support View.CreatePCollectionView in FlinkRunner
> -------------------------------------------------
>
>                 Key: BEAM-2806
>                 URL: https://issues.apache.org/jira/browse/BEAM-2806
>             Project: Beam
>          Issue Type: New Feature
>          Components: runner-flink
>            Reporter: Xu Mingmin
>            Assignee: Aljoscha Krettek
>
> Beam version: 2.2.0-SNAPSHOT
> Here's the code
> {code}
> PCollectionView<Map<BeamRecord, Iterable<BeamRecord>>> rowsView = rightRows
>         .apply(View.<BeamRecord, BeamRecord>asMultimap());
> {code}
> And exception when running with {{FlinkRunner}}:
> {code}
> Exception in thread "main" java.lang.UnsupportedOperationException: The transform View.CreatePCollectionView
is currently not supported.
> 	at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform(FlinkStreamingPipelineTranslator.java:113)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:594)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:586)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:586)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500(TransformHierarchy.java:268)
> 	at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:202)
> 	at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:440)
> 	at org.apache.beam.runners.flink.FlinkPipelineTranslator.translate(FlinkPipelineTranslator.java:38)
> 	at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate(FlinkStreamingPipelineTranslator.java:69)
> 	at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:104)
> 	at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
> 	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
> 	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:283)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message