flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6188) Some setParallelism() method can't cope with default parallelism
Date Sat, 25 Mar 2017 21:39:42 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941990#comment-15941990
] 

Aljoscha Krettek commented on FLINK-6188:
-----------------------------------------

This is the thrown exception:
{code}
Caused by: java.lang.IllegalArgumentException: The parallelism of an operator must be
at least 1.
	at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:139)
	at org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator.setParallelism(SingleOutputStreamOperator.java:124)
	at org.apache.flink.streaming.api.datastream.DataStream.assignTimestampsAndWatermarks(DataStream.java:775)
	at org.apache.flink.streaming.api.scala.DataStream.assignTimestampsAndWatermarks(DataStream.scala:736)
	...
{code}

The problem is that {{assignTimestampsAndWatermarks()}} uses {{setParallelism()}} with the
parallelism of the upstream operation, which can be {{-1}}.

> Some setParallelism() method can't cope with default parallelism
> ----------------------------------------------------------------
>
>                 Key: FLINK-6188
>                 URL: https://issues.apache.org/jira/browse/FLINK-6188
>             Project: Flink
>          Issue Type: Bug
>          Components: DataStream API
>    Affects Versions: 1.2.1
>            Reporter: Aljoscha Krettek
>            Priority: Blocker
>             Fix For: 1.2.1
>
>
> Recent changes done for FLINK-5808 move default parallelism manifestation from eager
to lazy, that is, the parallelism of operations that don't have an explicit parallelism is
only set when generating the JobGraph. Some `setParallelism()` calls, such as `SingleOutputStreamOperator.setParallelism()`
cannot deal with the fact that the parallelism of an operation might be {{-1}} (which indicates
that it should take the default parallelism when generating the JobGraph).
> We should either revert the changes that fixed another user-facing bug for version 1.2.1
or fix the methods.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message