flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominique Rondé <dominique.ro...@allsecur.de>
Subject Restart Flink in Yarn
Date Wed, 04 May 2016 16:27:06 GMT
Hi @all,

i have a yarn cluster with 5 Nodes with a running flink (0.10.2) 
instance. Today we shut down one of the Yarn-Hosts due to maintance 
reasons. After the restart we have some flink streaming routes in a 
restarting status (see stacktrace below). Now I want to restart these 
routes to continue their work from the last checkpoint. What can i do?

Greets
Dominique

Stacktrace
===================================================================================

java.io.IOException: Cannot get library with hash 8f15fe4a8137ca2f9fb348ec634f3703f4fd7317
	at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager.registerReferenceToBlobKeyAndGetURL(BlobLibraryCacheManager.java:254)
	at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager.registerTask(BlobLibraryCacheManager.java:114)
	at org.apache.flink.runtime.taskmanager.Task.createUserCodeClassloader(Task.java:710)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:471)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Failed to fetch BLOB 8f15fe4a8137ca2f9fb348ec634f3703f4fd7317
from /10.24.20.14:60485 and store it under /tmp/blobStore-efdeddf9-d096-440f-a4cb-9c79334ff92c/cache/blob_8f15fe4a8137ca2f9fb348ec634f3703f4fd7317
	at org.apache.flink.runtime.blob.BlobCache.getURL(BlobCache.java:177)
	at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager.registerReferenceToBlobKeyAndGetURL(BlobLibraryCacheManager.java:245)
	... 4 more
Caused by: java.io.IOException: GET operation failed: Server side error: Cannot find required
BLOB at /tmp/blobStore-0f9a63e3-5700-4d47-aea7-310506c1496c/cache/blob_8f15fe4a8137ca2f9fb348ec634f3703f4fd7317
	at org.apache.flink.runtime.blob.BlobClient.get(BlobClient.java:165)
	at org.apache.flink.runtime.blob.BlobCache.getURL(BlobCache.java:125)
	... 5 more
Caused by: java.io.IOException: Server side error: Cannot find required BLOB at /tmp/blobStore-0f9a63e3-5700-4d47-aea7-310506c1496c/cache/blob_8f15fe4a8137ca2f9fb348ec634f3703f4fd7317
	at org.apache.flink.runtime.blob.BlobClient.receiveAndCheckResponse(BlobClient.java:213)
	at org.apache.flink.runtime.blob.BlobClient.get(BlobClient.java:159)
	... 6 more
Caused by: java.io.IOException: Cannot find required BLOB at /tmp/blobStore-0f9a63e3-5700-4d47-aea7-310506c1496c/cache/blob_8f15fe4a8137ca2f9fb348ec634f3703f4fd7317
	at org.apache.flink.runtime.blob.BlobServerConnection.get(BlobServerConnection.java:202)
	at org.apache.flink.runtime.blob.BlobServerConnection.run(BlobServerConnection.java:112)



Mime
View raw message