flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yun Gao" <yungao...@aliyun.com>
Subject Re: Is there any way to get the ExecutionGraph of a Job
Date Tue, 04 Jun 2019 03:06:53 GMT
Hi Debasish,

     You cannot get ExecutionGraph since it resides in the JobMaster, which is not in the
same process with Client.

     In my opinion, currently you may not be able to stop the job or query the job status
using the Client API. The community is currently trying to enhance the Client API[1] so it
will be available in the future.  

     An alternative option may be using the REST API to monitor the status of the job, and
canceling job by calling the undocumented REST API <JM_ADDR>/jobs/<job id>/yarn-cancel
(May be removed in the future).

   [1] https://lists.apache.org/list.html?dev@flink.apache.org:lte=1M:Flink%20client%20api%20enhancement%20for%20downstream%20project
   [2] https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/rest_api.html

Yun Gao

From:Debasish Ghosh <ghosh.debasish@gmail.com>
Send Time:2019 Jun. 3 (Mon.) 17:32
To:user <user@flink.apache.org>
Subject:Is there any way to get the ExecutionGraph of a Job

Hello -

I am trying to build an API that can start, control and stop a Flink Job programmatically.

When I do an executionEnv.execute() where executionEnv is an StreamExecutionEnvironment, I
get back a JobExecutionResult. I find no way to stop the job (say based on some timeout) from
a JobExecutionResult.

One class that can help me is ExecutionGraph which has APIs like stop() and scheduleForExecution().
But how do I get an ExecutionGraph from executionEnv.execute(). All I want is to submit the
job for execution and then be able to stop it after some timeout. Or maybe if the job fails
I would like to do some cleanups. 

What is the idiomatic way to design such APIs in Flink ?

Debasish Ghosh

Twttr: @debasishg
Blog: http://debasishg.blogspot.com
Code: http://github.com/debasishg

View raw message