flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aljoscha <...@git.apache.org>
Subject [GitHub] flink pull request #3509: [FLINK-5808] Fix Missing verification for setParal...
Date Mon, 13 Mar 2017 14:25:08 GMT
Github user aljoscha commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3509#discussion_r105671778
  
    --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/StreamContextEnvironment.java
---
    @@ -38,14 +38,10 @@
     	private final ContextEnvironment ctx;
     
     	protected StreamContextEnvironment(ContextEnvironment ctx) {
    +		super(GlobalConfiguration.loadConfiguration().getInteger(
    --- End diff --
    
    This is not introducing the global configuration call. (Interestingly, this is one of
the oldest portions of code in the streaming API: https://github.com/apache/flink/commit/6867f9b#diff-615270fa9777c5c18e374b0269e358e5,
more than 2 years old 😉)
    
    I agree that there are some problems in how the streaming execution environments work
but I think we cannot easily get rid of that right now. The call graph is roughly `user main()
-> StreamExecutionEnvironment.getEnv() -> create context env` so there is nothing on
that path that would allow us to get the default parallelism. What we could do is have the
Flink client program put the default parallelism in the environment so that we can pick it
up here. This also seems highly hacky, though.
    
    I think the solution to this is to remove the default parallelism parameter from the config.
It seems kind of magical how that actually ends up in a user program. Both for the batch and
streaming APIs.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message