beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-153) Support timeout in runner API
Date Tue, 29 Mar 2016 18:56:25 GMT
Eugene Kirpichov created BEAM-153:
-------------------------------------

             Summary: Support timeout in runner API
                 Key: BEAM-153
                 URL: https://issues.apache.org/jira/browse/BEAM-153
             Project: Beam
          Issue Type: Improvement
            Reporter: Eugene Kirpichov


Some users want to make sure that their pipeline doesn't run longer than X minutes (e.g. because
sometimes it runs longer than that due to bugs, and in that case they'd rather auto-cancel
it than incur the costs).

The runner API should have a timeout option, so that if the pipeline isn't in a terminal state
by then, it is automatically cancelled.

Naturally, this only applies to batch pipelines.

A simple way to implement this for a blocking runner (such as BlockingDataflowPipelineRunner)
is a wrapper of the sort "start pipeline, and cancel it after timeout" inside run(). For a
non-blocking runner this will require support on the underlying execution environment side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message