flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Janardhan Reddy <janardhan.re...@olacabs.com>
Subject Job Stuck in cancel state
Date Mon, 19 Sep 2016 17:45:13 GMT
HI,

I cancelled a restarting job from flink UI and the job is stuck in
cancelling state. (Fixed delay restart strategy was configured for the
job). The following error message is present in taskmanager logs.

akka.remote.OversizedPayloadException: Discarding oversized payload sent to
Actor[akka.tcp://flink@10.200.7.245:42589/user/jobmanager#-146176374]: max
allowed size 10485760 bytes, actual size of encoded class
org.apache.flink.runtime.messages.JobManagerMessages$LeaderSessionMessage
was 20670224 bytes.


Does the leader session message here denote the job cancel message which is
sent to the job manager decorated with leader session id ?

Thread dump of taskmanager:


Attaching to process ID 28948, please wait...

Debugger attached successfully.

Server compiler detected.

JVM version is 25.101-b13

Deadlock Detection:


java.lang.RuntimeException: Unable to deduce type of thread from address
0x00007f96e56c9800 (expected type JavaThread, CompilerThread,
ServiceThread, JvmtiAgentThread, or SurrogateLockerThread)

at sun.jvm.hotspot.runtime.Threads.createJavaThreadWrapper(Threads.java:169)

at sun.jvm.hotspot.runtime.Threads.first(Threads.java:153)

at
sun.jvm.hotspot.runtime.DeadlockDetector.createThreadTable(DeadlockDetector.java:149)

at sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:56)

at sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:39)

at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:62)

at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)

at sun.jvm.hotspot.tools.JStack.run(JStack.java:66)

at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)

at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)

at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)

at sun.jvm.hotspot.tools.JStack.main(JStack.java:92)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at sun.tools.jstack.JStack.runJStackTool(JStack.java:140)

at sun.tools.jstack.JStack.main(JStack.java:106)

Caused by: sun.jvm.hotspot.types.WrongTypeException: No suitable match for
type of address 0x00007f96e56c9800

at
sun.jvm.hotspot.runtime.InstanceConstructor.newWrongTypeException(InstanceConstructor.java:62)

at
sun.jvm.hotspot.runtime.VirtualConstructor.instantiateWrapperFor(VirtualConstructor.java:80)

at sun.jvm.hotspot.runtime.Threads.createJavaThreadWrapper(Threads.java:165)

... 17 more

Can't print deadlocks:Unable to deduce type of thread from address
0x00007f96e56c9800 (expected type JavaThread, CompilerThread,
ServiceThread, JvmtiAgentThread, or SurrogateLockerThread)

Exception in thread "main" java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at sun.tools.jstack.JStack.runJStackTool(JStack.java:140)

at sun.tools.jstack.JStack.main(JStack.java:106)

Caused by: java.lang.RuntimeException: Unable to deduce type of thread from
address 0x00007f96e56c9800 (expected type JavaThread, CompilerThread,
ServiceThread, JvmtiAgentThread, or SurrogateLockerThread)

at sun.jvm.hotspot.runtime.Threads.createJavaThreadWrapper(Threads.java:169)

at sun.jvm.hotspot.runtime.Threads.first(Threads.java:153)

at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:75)

at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)

at sun.jvm.hotspot.tools.JStack.run(JStack.java:66)

at sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:260)

at sun.jvm.hotspot.tools.Tool.start(Tool.java:223)

at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)

at sun.jvm.hotspot.tools.JStack.main(JStack.java:92)

... 6 more

Caused by: sun.jvm.hotspot.types.WrongTypeException: No suitable match for
type of address 0x00007f96e56c9800

at
sun.jvm.hotspot.runtime.InstanceConstructor.newWrongTypeException(InstanceConstructor.java:62)

at
sun.jvm.hotspot.runtime.VirtualConstructor.instantiateWrapperFor(VirtualConstructor.java:80)

at sun.jvm.hotspot.runtime.Threads.createJavaThreadWrapper(Threads.java:165)

... 14 more

Mime
View raw message