flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "pengchenglins@163.com" <pengchengl...@163.com>
Subject Re: Flink集群迁移savepoint还保留原集群地址问题讨论
Date Thu, 10 Oct 2019 09:31:03 GMT
你好,生成savepoint之后,是否把savepoint移动到了其他地方?官方说不能移动。

 
发件人: 蒋涛涛
发送时间: 2019-10-10 17:13
收件人: user-zh
主题: Flink集群迁移savepoint还保留原集群地址问题讨论
HI all,
 
本来在Flink集群迁移过程中,使用版本flink
1.6.2,将savepoint进行保存,并迁移到新的hdfs集群的时候,并从该savepoint进行恢复的时候,出现找不到之前hdfs集群信息的错误信息,错误如下:
 
java.lang.Exception: Exception while creating StreamOperatorStateContext.
    at
org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:192)
    at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:250)
    at
org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:738)
    at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:289)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.flink.util.FlinkException: Could not restore operator
state backend for StreamSource_cbc357ccb763df2852fee8c4fc7d55f2_(1/1) from
any of the 1 provided restore options.
    at
org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:137)
    at
org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.operatorStateBackend(StreamTaskStateInitializerImpl.java:242)
    at
org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:140)
    ... 5 more
Caused by: java.io.IOException: Cannot instantiate file system for URI:
hdfs://xxx/user/flink_1.6/savepoints/savepoint-ab68c7-562a00439efe/2e56b78a-fa36-4cf4-a383-990358fcd539
    at
org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:187)
    at
org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:399)
    at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:318)
    at
org.apache.flink.runtime.state.filesystem.FileStateHandle.getFileSystem(FileStateHandle.java:100)
    at
org.apache.flink.runtime.state.filesystem.FileStateHandle.openInputStream(FileStateHandle.java:68)
    at
org.apache.flink.runtime.state.OperatorStreamStateHandle.openInputStream(OperatorStreamStateHandle.java:66)
    at
org.apache.flink.runtime.state.DefaultOperatorStateBackend.restore(DefaultOperatorStateBackend.java:495)
    at
org.apache.flink.runtime.state.DefaultOperatorStateBackend.restore(DefaultOperatorStateBackend.java:64)
    at
org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:151)
    at
org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:123)
    ... 7 more
Caused by: java.lang.IllegalArgumentException:
java.net.UnknownHostException: xxx
    at
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:439)
    at
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:321)
    at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:690)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:631)
    at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:160)
    at
org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:159)
    ... 16 more
Caused by: java.net.UnknownHostException: xxx
    ... 23 more
 
补充:
这个的xxx指的是原hdfs的clustername,用这个方式进行任务的迁移会出现这样的问题,checkpoint也有类似的错误,请问下为什么会出现这样的问题,并如何解决,谢谢
 
祝好
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message