flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Richter <s.rich...@data-artisans.com>
Subject Re: multiple flink applications on yarn are shown in one application.
Date Fri, 21 Sep 2018 08:15:33 GMT
Hi,

I see from your command that you are using the same jar file twice, so I want to double-check
first how you even determine which job should be started? I am also adding Till (in CC) depending
on your answer to my first question, he might have some additional thoughts.

Best,
Stefan

> Am 21.09.2018 um 04:15 schrieb weilongxing <weilongxing@aicaigroup.com>:
> 
> Thank you for your help.
> 
> It is per-job mode if I submit the job detached.
> 
> However, I submit these two jobs use the command below. 
> > /data/apps/opt/flink/bin/flink run -m yarn-cluster -d /data/apps/opt/fluxion/fluxion-flink.jar
/data/apps/conf/fluxion//job_submit_rpt_company_performan.conf
> > /data/apps/opt/flink/bin/flink run -m yarn-cluster -d /data/apps/opt/fluxion/fluxion-flink.jar
/data/apps/conf/fluxion//job_submit_rpt_company_user_s.conf
> 
> I browse the yarn application. As the picture shows I got 2 applications(0013 / 0012)
but the job in both applications is the same. I can’t find the job submitted secondly. The
job in application_XXX_0013 should be rpt_company_user_s. This will not happen in session
mode. 
> 
> Best
> LX
> 
> 
> 
> <PastedGraphic-13.png>
> <PastedGraphic-14.png>
> 
>> 在 2018年9月20日,下午7:13,Stefan Richter <s.richter@data-artisans.com
<mailto:s.richter@data-artisans.com>> 写道:
>> 
>> Hi,
>> 
>> currently, Flink still has to use session mode under the hood if you submit the job
in attached-mode. The reason is that the job could consists of multiple parts that require
to run one after the other. This will be changed in the future and also should not happen
if you
>> submit the job detached.
>> 
>> Best,
>> Stefan
>> 
>>> Am 20.09.2018 um 10:29 schrieb weilongxing <weilongxing@aicaigroup.com <mailto:weilongxing@aicaigroup.com>>:
>>> 
>>> I am new to Flink. I am using Flink on yarn per-job. I submitted two applications.
>>> 
>>> > /data/apps/opt/flink/bin/flink run -m yarn-cluster  /data/apps/opt/fluxion/fluxion-flink.jar
/data/apps/conf/fluxion//job_submit_rpt_company_user_s.conf
>>> 
>>> > /data/apps/opt/flink/bin/flink run -m yarn-cluster  /data/apps/opt/fluxion/fluxion-flink.jar
/data/apps/conf/fluxion//job_submit_rpt_company_performan_.conf
>>> 
>>> I can saw these two applications on yarn. I noticed that the application name
is “flink session cluster” rather than “flink per-job”. Is that right?
>>> <PastedGraphic-10.png>
>>> 
>>> However, I can see both flink jobs in each yarn application. Is that right?
>>> 
>>> 
>>> <PastedGraphic-8.png>
>>> <PastedGraphic-9.png>
>>> 
>>> 
>>> And Finally, I want to kill one job and killed one yarn application. However
one yarn application is killed and but both flink jobs restarted in another yarn application.
I want to kill one and remain another. In my opinion, one job in an application and the job
is killed when the yarn application is killed.
>>> 
>>> <PastedGraphic-12.png>
>>> 
>>> 
>>> I think the problem is that these two application should be “flink per-job”
rather than “flink session cluster”. But I don’t know why it becomes “flink session-cluster”.
 Can anybody help? Thanks.
>>> 
>>> 
>>> 
>> 
> 


Mime
View raw message