flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang Wang <danrtsey...@gmail.com>
Subject Re: Flink on yarn use jar on hdfs
Date Mon, 16 Sep 2019 06:44:16 GMT
Hi Shengnan,

I think you mean to avoid uploading flink-dist jars in submission every
time.
I have created a JIRA[1] to use Yarn public cache to speed up the launch
duration of JM and TM. After this feature merged, you could submit a flink
job like below.

./bin/flink run -d -m yarn-cluster -p 20 -ysl
hdfs:///flink/release/flink-1.9.0/lib
examples/streaming/WindowJoin.jar


[1]. https://issues.apache.org/jira/browse/FLINK-13938

Shengnan YU <ysnakie@hotmail.com> 于2019年9月16日周一 下午2:24写道:

> Hi everyone!
> I found that everytime I start a flink-yarn application, client will ship
> flink-uber jar and other dependencies to hdfs and start appMaster. Is there
> any approaches to locate flink-uber jar and other library jars on hdfs and
> let only configuration file being shipped. Therefore the yarn can start
> flink appMaster using jar on a fixed location in hdfs? Thank you very much!
>
>

Mime
View raw message