flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject execute() and collect()/print()/count()
Date Fri, 19 Jun 2015 12:15:19 GMT
Dear Flink community,

I have stopped to count how many people on the user list and during Flink
trainings have asked why their Flink program throws an Exception when they
just one to print a DataSet. The reason for this is that print() now
executes eagerly, thus, executes the Flink program. Subsequent calls to
execute() need to define new DataSinks and throw an exception otherwise.

We have recently introduced a flag in the ExecutionEnvironment that checks
whether the user executed before (explicitly via execute() or implicitly
through collect()/print()/count()). That enabled us to print a nicer
exception message. However, users either do not read the exception message
or do not understand it. They do ask this question a lot.

That's why I propose to ignore calls to execute() entirely if no sinks are
defined. That will get rid of one of the core annoyances for Flink users. I
know, that this is painfully for us programmers because we understand how
Flink works internally but let's step back once and see that it wouldn't be
so bad if execute didn't do anything in case of no new sinks.

What would be the downside of this change? Users might call execute() and
wonder that nothing happens. We would then simply print a warning that
their program didn't define any sinks. That is a big difference to the
behavior before because users are scared of exceptions. If they just get a
warning they will double-check their program and investigate why nothing
happens. Most of the cases they do actually have defined sinks but simply
left a call to execute() when they were printing a DataSet.

What are you opinions on this issue? I have opened a JIRA for this as well:
https://issues.apache.org/jira/browse/FLINK-2249

Best,
Max

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message