flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tillrohrmann <...@git.apache.org>
Subject [GitHub] flink pull request: [FLINK-2111] Add "stop" signal to cleanly shut...
Date Fri, 05 Feb 2016 23:20:02 GMT
Github user tillrohrmann commented on the pull request:

    Ok let us include as a first version a stop implementation where you don't have the strong
consistency guarantees. But before I think we should still change three things.
    The first thing I would like to change is the `JobType`. I feel a bit uneasy adding explicit
information whether a job is a streaming or a batch job to the `JobGraph`. So far we've succeeded
in keeping the runtime agnostic to the type of job. I would propose to change it to a `stoppable`
flag. Thus, indicating whether the job is stoppable or not. Semantically, it should not make
a big difference.
    The second thing I would propose is to remove `stop` from the `StreamSource` class in
order to make it not mandatory for all streaming sources to be stoppable. Instead I would
also introduce a `StoppableOperator` interface which can be implemented by streaming sources
which are in fact stoppable. I'm not so sure, for example, whether the Kafka sources are properly
stoppable without throwing exceptions. By introducing this interface we can selectively implement
stop methods for sources. A nice side effect is that existing code won't break if you have
a custom source implementation.
    And as a last point I would propose to automatically infer whether a job is stoppable
or not. What we have to do is to check that all source functions implement the `StoppableOperator`
interface. This should be doable in the `StreamGraphGenerator`. Based on this information
we can then set the stoppable flag in the `JobGraph` or not.
    What do you think? Would that be feasible?

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message