beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ismaël Mejía (JIRA) <j...@apache.org>
Subject [jira] [Commented] (BEAM-3589) Flink runner breaks with ClassCastException on UnboundedSource
Date Thu, 01 Feb 2018 13:41:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348582#comment-16348582
] 

Ismaël Mejía commented on BEAM-3589:
------------------------------------

Great work [~grzegorz_kolakowski] your fix works perfectly, we should however get it merged
and cherry-pick it into version 2.3.0. I will let the ticket open until this is done.

> Flink runner breaks with ClassCastException on UnboundedSource
> --------------------------------------------------------------
>
>                 Key: BEAM-3589
>                 URL: https://issues.apache.org/jira/browse/BEAM-3589
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 2.3.0
>            Reporter: Ismaël Mejía
>            Assignee: Grzegorz Kołakowski
>            Priority: Blocker
>
> When you execute a pipeline tha uses an unbounded source and an empty transform it produces
a ClassCastException:
> {quote}[WARNING] 
> java.lang.reflect.InvocationTargetException
>     at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke (Method.java:498)
>     at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
>     at java.lang.Thread.run (Thread.java:748)
> Caused by: java.lang.ClassCastException: org.apache.beam.runners.core.construction.AutoValue_PTransformTranslation_UnknownRawPTransform
cannot be cast to org.apache.beam.sdk.io.Read$Unbounded
>     at org.apache.beam.runners.flink.FlinkStreamingTransformTranslators$ReadSourceTranslator.translateNode
(FlinkStreamingTransformTranslators.java:256)
>     at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.applyStreamingTransform
(FlinkStreamingPipelineTranslator.java:139)
>     at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.visitPrimitiveTransform
(FlinkStreamingPipelineTranslator.java:118)
>     at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:670)
>     at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:647)
>     at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:623)
>     at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:623)
>     at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:647)
>     at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit (TransformHierarchy.java:662)
>     at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600 (TransformHierarchy.java:311)
>     at org.apache.beam.sdk.runners.TransformHierarchy.visit (TransformHierarchy.java:245)
>     at org.apache.beam.sdk.Pipeline.traverseTopologically (Pipeline.java:458)
>     at org.apache.beam.runners.flink.FlinkPipelineTranslator.translate (FlinkPipelineTranslator.java:38)
>     at org.apache.beam.runners.flink.FlinkStreamingPipelineTranslator.translate (FlinkStreamingPipelineTranslator.java:70)
>     at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate (FlinkPipelineExecutionEnvironment.java:113)
>     at org.apache.beam.runners.flink.FlinkRunner.run (FlinkRunner.java:110)
>     at org.apache.beam.sdk.Pipeline.run (Pipeline.java:311)
>     at org.apache.beam.sdk.Pipeline.run (Pipeline.java:297)
>     at org.apache.beam.sdk.nexmark.NexmarkLauncher.run (NexmarkLauncher.java:1139)
>     at org.apache.beam.sdk.nexmark.Main.runAll (Main.java:69)
>     at org.apache.beam.sdk.nexmark.Main.main (Main.java:301)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke (Method.java:498)
>     at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:294)
>     at java.lang.Thread.run (Thread.java:748)
> {quote}
> You can reproduce it quickly by running this command from the nexmark directory:
> {quote}mvn exec:java -Dexec.mainClass=org.apache.beam.sdk.nexmark.Main -Pflink-runner
"-Dexec.args=--runner=FlinkRunner --query=2 --streaming=true --manageResources=false --monitorJobs=true"
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message