flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2318) BroadcastVariable of unioned data set fails
Date Tue, 24 Nov 2015 21:49:11 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025495#comment-15025495
] 

Fabian Hueske commented on FLINK-2318:
--------------------------------------

I think it would be good to have this fix also into the {{release-0.10}} branch.

> BroadcastVariable of unioned data set fails
> -------------------------------------------
>
>                 Key: FLINK-2318
>                 URL: https://issues.apache.org/jira/browse/FLINK-2318
>             Project: Flink
>          Issue Type: Bug
>          Components: Distributed Runtime, Optimizer
>    Affects Versions: 0.9
>            Reporter: Fabian Hueske
>            Assignee: Chesnay Schepler
>
> Using a unioned data set as broadcast variable such as this:
> {code}
> DataSet d1 = [...]
> DataSet d2 = [...]
> DataSet d3 = [...]
> d1
>   .map(new MyMapper())
>   .withBroadcastSet(d2.union(d3), "myBroadcast");
> {code}
> throws an exception at runtime:
> {code}
> java.lang.Exception: Call to registerInputOutput() of invokable failed
> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:504)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: Initializing the input streams failed in Task
MapPartition (MapPartition at translatHashJoinAsMap(FlinkFlowStep.java:755)): Illegal input
group size in task configuration: -1
> 	at org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:246)
> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:501)
> 	... 1 more
> Caused by: java.lang.Exception: Illegal input group size in task configuration: -1
> 	at org.apache.flink.runtime.operators.RegularPactTask.initBroadcastInputReaders(RegularPactTask.java:783)
> 	at org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:243)
> 	... 2 more
> {code}
> A simple workaround is to apply an identity mapper on the unioned data set.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message