flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arvid Heise <ar...@ververica.com>
Subject Re: Retrieving Flink job ID/YARN Id programmatically
Date Thu, 28 Nov 2019 12:35:52 GMT
Hi Lei,

if you use

public JobExecutionResult StreamExecutionEnvironment#execute()

You can retrieve the job id through the result.

result.getJobID()

Best,

Arvid

On Mon, Nov 25, 2019 at 3:50 AM Ana <otakunoneko@gmail.com> wrote:

> Hi Lei,
>
> To add, you may use Hadoop Resource Manager REST APIs
> https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html.
> I'm also running Flink application on YARN and use this API for that
> purpose. If you find other way or a much better solution, please let me
> know!
>
> Regards,
> Ana
>
> On Fri, Nov 22, 2019 at 10:58 AM vino yang <yanghua1127@gmail.com> wrote:
>
>> Hi Lei,
>>
>> It would be better to use Flink's RESTful API to fetch the information of
>> the running jobs[1].
>>
>> [1]:
>> https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/rest_api.html#jobs-1
>>
>> Best,
>> Vino
>>
>> Lei Nie <lyzjn81@gmail.com> 于2019年11月22日周五 上午4:14写道:
>>
>>> I looked at the code, and
>>> StreamExecutionEnvironment#getStreamGraph#getJobGraph#getJobID is
>>> generating a random ID unrelated to the actual ID used.
>>>
>>> Is there any way to fetch the real ID at runtime?
>>> Use case: fetch most recent checkpoint from stable storage for
>>> automated restarts. Most recent checkpoint has form
>>> ".../checkpoints/flink_app_id/chk-123"
>>>
>>> On Thu, Nov 21, 2019 at 11:28 AM Lei Nie <lyzjn81@gmail.com> wrote:
>>> >
>>> > This does not get the correct id:
>>> > StreamExecutionEnvironment#getStreamGraph#getJobGraph#getJobID =
>>> > eea5abc21dd8743a4090f4a3a660f9e8
>>> > Actual job ID (from webUI): 1357d21be640b6a3b8a86a063f4bba8a
>>> >
>>> >
>>> >
>>> > On Thu, Nov 7, 2019 at 6:56 PM vino yang <yanghua1127@gmail.com>
>>> wrote:
>>> > >
>>> > > Hi Lei Nie,
>>> > >
>>> > > You can use
>>> `StreamExecutionEnvironment#getStreamGraph#getJobGraph#getJobID` to get the
>>> job id.
>>> > >
>>> > > Best,
>>> > > Vino
>>> > >
>>> > > Lei Nie <lyzjn81@gmail.com> 于2019年11月8日周五 上午8:38写道:
>>> > >>
>>> > >> Hello,
>>> > >> I am currently executing streaming jobs via
>>> StreamExecutionEnvironment. Is it possible to retrieve the Flink job
>>> ID/YARN ID within the context of a job? I'd like to be able to
>>> automatically register the job such that monitoring jobs can run (REST api
>>> requires for example job id).
>>> > >>
>>> > >> Thanks
>>>
>>

Mime
View raw message