flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weihua Hu <huweihua....@gmail.com>
Subject Re: 关于flink run -m yarn提交失败。flink1.9
Date Fri, 17 Apr 2020 03:52:55 GMT
Hi, dlguanyq

Deployment took more than 60 seconds.
——
这个日志表示已经将 Application 提交到 Yarn 了,但是 AM 一直没有启动,这一步和perjob模式或者
yarn-session 模式关系不大。
可以用 -yd 多试几次,还是不能成功的话,需要检查下 yarn 的日志

Best
Weihua Hu

> 2020年4月15日 09:06,guanyq <dlguanyq@163.com> 写道:
> 
> 使用的是perjob模式提交作业,没有使用yarn-seesion。为什么perjob模式提交有这个-yd参数会有问题,还是没太懂。
> 在 2020-04-15 08:52:11,"tison" <wander4096@gmail.com> 写道:
>> -yd 参数影响的是你是否使用 perjob 模式提交作业,简单地说
>> 
>> with -yd 以 perjob 模式提交作业,即启动一个新集群
>> without -yd 提交到一个现有的 Flink on YARN 集群
>> 
>> 哪个是你的需求呢?有没有实现用 yarn-session 启动 Flink on YARN 集群呢?
>> 
>> Best,
>> tison.
>> 
>> 
>> guanyq <dlguanyq@163.com> 于2020年4月15日周三 上午8:46写道:
>> 
>>> 提交失败,本人测试与-yd参数有关系,<br/>这个参数去掉就可以提交了。<br/>但是不知道
-yd这个参数影响了什么?
>>> At 2020-04-14 15:31:00, "guanyq" <dlguanyq@163.com> wrote:
>>>> 提交失败,yarn资源也还有很多,为什么会提交失败呢?
>>>> 
>>>> 提交脚本
>>>> ./bin/flink run -m yarn-cluster \
>>>> -ynm TestDataProcess \
>>>> -yd \
>>>> -yn 2 \
>>>> -ytm 1024 \
>>>> -yjm 1024 \
>>>> -c com.data.processing.unconditionalacceptance.TestDataProcess \
>>>> ./tasks/UnconditionalAcceptanceDataProcess.jar \
>>>> 
>>>> 
>>>> yarn资源
>>>> Apps Submitted Apps Pending    Apps Running    Apps Completed  Containers
>>> Running      Memory Used     Memory Total    Memory Reserved VCores Used
>>> VCores Total    VCores Reserved Active Nodes    Decommissioned Nodes
>>> Lost Nodes      Unhealthy Nodes Rebooted Nodes
>>>> 239    0       12      227     173     334 GB  1.42 TB 0 B     173
>>> 288     0       9       0       0       0       0
>>>> 
>>>> 
>>>> 
>>>> 2020-04-14 15:14:19,002 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:19,253 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:19,504 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:19,755 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:20,006 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:20,257 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:20,508 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:20,759 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:21,011 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:21,262 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:21,513 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:21,764 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:22,015 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:22,265 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:22,517 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:22,768 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:23,019 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:23,270 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:23,521 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:23,772 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:24,025 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>>> 2020-04-14 15:14:24,276 INFO
>>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>>> took more than 60 seconds. Please check if the requested resources are
>>> available in the YARN cluster
>>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message