hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: HRegionServer not shutting down
Date Mon, 12 Sep 2011 15:41:44 GMT
Hey Jeff,

Usually if jstack gives you such an error, you may try:

$ sudo -u hbase jstack <PID> # Or sudo to whatever the process owner is.

The -F opt is unhelpful at times, as in this case (don't see any useful info in the output
- just thread numbers and exceptions everywhere).

P.s. There's also a new bin/graceful_stop.sh script to stop RegionServers elegantly in HBase
0.90 now. Not exactly related to your problem perhaps, but thought you might wanna know!

On 12-Sep-2011, at 8:54 PM, Jeff Whiting wrote:

> We are running cdh3u1 on centos 5.6 and we have frequent problems shutting down region
servers.  They get stuck and the only thing I can do to stop them is a kill -9 which is a
little harsh.  I tried doing a normal jstack but it doesn't respond.  With the -F option I
get what you see at the bottom of the email.  Also in the region server logs I see the following
which would imply that it should be shutdown.
> 
> 2011-09-11 13:16:18,692 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump
of metrics: requests=0, regions=16, stores=27, storefiles=17, storefileIndexSize=1, memstoreSize=10,
compactionQueueSize=0, flushQueueSize=0, usedHeap=609, maxHeap=795, blockCacheSize=130088528,
blockCacheFree=36818560, blockCacheCount=1055, blockCacheHitCount=2038243, blockCacheMissCount=92093,
blockCacheEvictedCount=600, blockCacheHitRatio=95, blockCacheHitCachingRatio=99
> 2011-09-11 13:16:18,692 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
connection to cluster: dev2-0x331633b7ddb08ef-0x331633b7ddb08ef-0x331633b7ddb08ef connection
to cluster: dev2-0x331633b7ddb08ef-0x331633b7ddb08ef-0x331633b7ddb08ef received expired from
ZooKeeper, aborting
> 2011-09-11 13:16:18,692 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
> 2011-09-12 09:07:16,289 INFO org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown
hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-15,5,main]
> 2011-09-12 09:07:16,289 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
Shutdown hook
> Mon Sep 12 09:14:26 MDT 2011 Killing regionserver
> 
> Any ideas as to why the region servers can't shutdown? My java version is:
> 
> [root@ds1 hbase]# java -version
> java version "1.6.0_21"
> Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
> Java HotSpot(TM) 64-Bit Server VM (build 17.0-b16, mixed mode)
> 
> Thanks,
> 
> ~Jeff
> 
> [root@ds1 ~]# jstack 18789
> 18789: Unable to open socket file: target process not responding or HotSpot VM not loaded
> The -F option can be used when the target process is not responding
> [root@ds1 ~]# jstack -F 18789
> Attaching to process ID 18789, please wait...
> Debugger attached successfully.
> Server compiler detected.
> JVM version is 17.0-b16
> Deadlock Detection:
> 
> No deadlocks found.
> 
> Thread 17578: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 17322: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 16419: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 16417: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 19017: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18901: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18871: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18870: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18815: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18868: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18843: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18840: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18823: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18822: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18821: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> Thread 18820: (state = BLOCKED)
> Error occurred during stack walking:
> sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException:
get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
>    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
>    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
>    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
>    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
>    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
>    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
>    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
>    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
>    at sun.tools.jstack.JStack.main(JStack.java:84)
> Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native
Method)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
>    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)
> 
> 
> -- 
> Jeff Whiting
> Qualtrics Senior Software Engineer
> jeffw@qualtrics.com
> 


Mime
View raw message