flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yun Tang <myas...@live.com>
Subject Re: blink(基于flink1.5.1版本)可以使用两个hadoop集群吗?
Date Sun, 26 Jan 2020 16:51:56 GMT
Hi Yong

首先,这封邮件就不要抄送开发者邮件列表了,中文的邮件只需要发中文邮件列表。

Flink当然可以用两个YARN集群,关键在于Flink提交作业到YARN的时候,读取的HADDOP配置是什么,其实官方文档[1]
有相关的介绍,主要是 YARN_CONF_DIR, HADOOP_CONF_DIR or HADOOP_CONF_PATH 这些环境变量的配置是什么,在你提交的终端内配置一个你搭建的集群环境变量即可。

[1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/yarn_setup.html#background--internals

祝好
唐云
________________________________
From: Yong <lujinyong168@foxmail.com>
Sent: Wednesday, January 22, 2020 14:53
To: dev <dev@flink.apache.org>; user-zh <user-zh@flink.apache.org>
Subject: blink(基于flink1.5.1版本)可以使用两个hadoop集群吗?

大家好,
&nbsp; &nbsp; flink可以使用两个hadoop集群吗?
背景如下:
&nbsp; &nbsp; 目前我这边基于blink搭建了flink standalone集群,状态存储使用公司的hadoop
hdfs 并且使用了kerberos认证,目前是可以正常运行,这个模式需要事先预留一部分资源(TM)保证job故障转移。考虑后续集群扩大,为了节约资源,想使用Hadoop
YARN来管理flink资源,但是公司的YARN不对外开放,所以自己搭建了一个小的Hadoop
YARN来管理flink资源,测试时候发现还是走了公司的hadoop集群了,该如何修改配置使用自己的YARN呢?
报错如下:
[qateadmin@UAT14475 bin]$ ./yarn-session.sh
----忽略kerberos日志信息----
2020-01-22 14:47:36,993 INFO&nbsp; org.apache.hadoop.security.UserGroupInformation&nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;- Login successful
for user htlapidev@DC.SH.CTRIPCORP.COM using keytab file /opt/data/blink/htlapidev.keytab
2020-01-22 14:47:36,994 INFO&nbsp; org.apache.flink.runtime.security.modules.HadoopModule&nbsp;
&nbsp; &nbsp; &nbsp; - Hadoop user set to htlapidev@DC.SH.CTRIPCORP.COM (auth:KERBEROS)
2020-01-22 14:47:37,296 INFO&nbsp; org.apache.flink.yarn.cli.FlinkYarnSessionCli&nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;- No
path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor
to locate the jar
2020-01-22 14:47:37,450 INFO&nbsp; org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider&nbsp;
- Failing over to rm2
2020-01-22 14:47:37,491 INFO&nbsp; org.apache.hadoop.io.retry.RetryInvocationHandler&nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;- Exception while invoking
getClusterNodes of class ApplicationClientProtocolPBClientImpl over rm2 after 1 fail over
attempts. Trying to fail over immediately.
java.io.IOException: Failed on local exception: java.io.IOException: Server asks us to fall
back to SIMPLE auth, but this client is configured to only allow secure connections.; Host
Details : local host is: "UAT14475/10.5.119.0"; destination host is: "uat14476":8032;
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.call(Client.java:1475)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.call(Client.java:1408)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
&nbsp; &nbsp; &nbsp; &nbsp; at com.sun.proxy.$Proxy16.getClusterNodes(Unknown
Source)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterNodes(ApplicationClientProtocolPBClientImpl.java:262)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&nbsp; &nbsp; &nbsp; &nbsp; at java.lang.reflect.Method.invoke(Method.java:498)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
&nbsp; &nbsp; &nbsp; &nbsp; at com.sun.proxy.$Proxy17.getClusterNodes(Unknown
Source)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(YarnClientImpl.java:488)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:318)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:539)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:448)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:659)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$7(FlinkYarnSessionCli.java:887)
&nbsp; &nbsp; &nbsp; &nbsp; at java.security.AccessController.doPrivileged(Native
Method)
&nbsp; &nbsp; &nbsp; &nbsp; at javax.security.auth.Subject.doAs(Subject.java:422)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:887)
Caused by: java.io.IOException: Server asks us to fall back to SIMPLE auth, but this client
is configured to only allow secure connections.
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:755)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.getConnection(Client.java:1524)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.call(Client.java:1447)
&nbsp; &nbsp; &nbsp; &nbsp; ... 22 more
2020-01-22 14:47:37,495 INFO&nbsp; org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider&nbsp;
- Failing over to rm1
2020-01-22 14:47:37,498 INFO&nbsp; org.apache.hadoop.io.retry.RetryInvocationHandler&nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;- Exception while invoking
getClusterNodes of class ApplicationClientProtocolPBClientImpl over rm1 after 2 fail over
attempts. Trying to fail over after sleeping for 36261ms.
java.net.ConnectException: Call From UAT14475/10.5.119.0 to uat14475.novalocal:8032 failed
on connection exception: java.net.ConnectException: Connection refused; For more details see:&nbsp;
http://wiki.apache.org/hadoop/ConnectionRefused
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
&nbsp; &nbsp; &nbsp; &nbsp; at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.call(Client.java:1475)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.call(Client.java:1408)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
&nbsp; &nbsp; &nbsp; &nbsp; at com.sun.proxy.$Proxy16.getClusterNodes(Unknown
Source)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterNodes(ApplicationClientProtocolPBClientImpl.java:262)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
&nbsp; &nbsp; &nbsp; &nbsp; at java.lang.reflect.Method.invoke(Method.java:498)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
&nbsp; &nbsp; &nbsp; &nbsp; at com.sun.proxy.$Proxy17.getClusterNodes(Unknown
Source)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(YarnClientImpl.java:488)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:318)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:539)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:448)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:659)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$7(FlinkYarnSessionCli.java:887)
&nbsp; &nbsp; &nbsp; &nbsp; at java.security.AccessController.doPrivileged(Native
Method)
&nbsp; &nbsp; &nbsp; &nbsp; at javax.security.auth.Subject.doAs(Subject.java:422)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:887)
Caused by: java.net.ConnectException: Connection refused
&nbsp; &nbsp; &nbsp; &nbsp; at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
&nbsp; &nbsp; &nbsp; &nbsp; at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:614)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:713)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.getConnection(Client.java:1524)
&nbsp; &nbsp; &nbsp; &nbsp; at org.apache.hadoop.ipc.Client.call(Client.java:1447)
&nbsp; &nbsp; &nbsp; &nbsp; ... 22 more
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message