db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Durie" <bob.du...@thirdbrigade.com>
Subject derby/jvm crashes
Date Thu, 27 Sep 2007 13:46:06 GMT
Hi,

 

We use derby as one option for database support for an enterprise server
product.  We're using derby 10.2.1.6 with the 1.5.0_11 jre, hotspot
server version.  Occasionally when our application is using derby (it
also supports oracle, sql server, and never seems to crash when using
those) we get crashes.  The jvm will dump hs_err file in the working
directory.  I've got a bunch of these sample files output below (just
the beginning of the output).

 

We cannot trace down a particular action within our application that
causes these, but low memory, VM environments seem to trigger it.
Sometimes right at startup, sometimes after running for a while.  Our
application has a fixed upper bounded jvm memory usage of 1 GB
(-Xmx1024m), but the VM's never have that much dedicated ram.  However,
we don't get OutOfMemory exceptions or anything anomalous leading up to
these crashes, they just happen.

 

The crashes are not always in a "CompilerThread" (crashes 1-3 below),
sometimes they are in our code too (see crash 4 below).  And then
sometimes they're not even in derby code (crash 5 and 6 below).

 

Below are a few sample crashes.  Does anyone have *any* suggests on what
I can try to do here?  I don't know where to begin!!!

 

--- CRASH1 ---

#

# An unexpected error has been detected by HotSpot Virtual Machine:

#

#  EXCEPTION_INT_DIVIDE_BY_ZERO (0xc0000094) at pc=0x6dac8e7e, pid=1600,
tid=1880

#

# Java VM: Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode)

# Problematic frame:

# V  [jvm.dll+0x1e8e7e]

#

 

---------------  T H R E A D  ---------------

 

Current thread (0x47f58e90):  JavaThread "CompilerThread0" daemon
[_thread_in_native, id=1880]

 

siginfo: ExceptionCode=0xc0000094

 

Registers:

EAX=0x00000000, EBX=0x48766a38, ECX=0x00000000, EDX=0x00000000

ESP=0x481ef0b8, EBP=0x488adfa4, ESI=0x00000000, EDI=0x00000015

EIP=0x6dac8e7e, EFLAGS=0x00010246

 

Top of Stack: (sp=0x481ef0b8)

0x481ef0b8:   481ef360 49edea80 481ef0f0 48766a78

0x481ef0c8:   49edea80 6dacc260 481ef360 49ee02c0

0x481ef0d8:   48763b14 48763b14 481ef360 48766a78

0x481ef0e8:   48766a38 48766a78 49edea80 6dacc48a

0x481ef0f8:   00000001 481ef134 49ee02c0 481ef360

0x481ef108:   4aafd408 00000000 6dacfa74 481ef360

0x481ef118:   481ef134 0000002a 481ef7d0 00000003

0x481ef128:   00000000 00000003 47f59020 47f59020 

 

Instructions: (pc=0x6dac8e7e)

0x6dac8e6e:   f8 77 37 8b f7 0f af f1 3b f0 77 2e 8b c6 33 d2

0x6dac8e7e:   f7 f1 3b f8 75 24 8b 44 24 18 8b 15 3c 6e bf 6d 

 

 

Stack: [0x481a0000,0x481f0000),  sp=0x481ef0b8,  free space=316k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code)

V  [jvm.dll+0x1e8e7e]

 

 

Current CompileTask:

opto:1825
org.apache.derby.impl.services.locks.LockControl.getLock(Ljava/lang/Obje
ct;Ljava/lang/Object;)Lorg/apache/derby/impl/services/locks/Lock; (102
bytes)

 

 

 

--- CRASH2 ---

#

# An unexpected error has been detected by HotSpot Virtual Machine:

#

#  EXCEPTION_INT_DIVIDE_BY_ZERO (0xc0000094) at pc=0x6dac8e7e, pid=1132,
tid=1572

#

# Java VM: Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode)

# Problematic frame:

# V  [jvm.dll+0x1e8e7e]

#

 

---------------  T H R E A D  ---------------

 

Current thread (0x47f099b8):  JavaThread "CompilerThread0" daemon
[_thread_in_native, id=1572]

 

siginfo: ExceptionCode=0xc0000094

 

Registers:

EAX=0x00000000, EBX=0x49c18720, ECX=0x00000000, EDX=0x00000000

ESP=0x481ef2b0, EBP=0x4920afa0, ESI=0x00000000, EDI=0x00000075

EIP=0x6dac8e7e, EFLAGS=0x00010246

 

Top of Stack: (sp=0x481ef2b0)

0x481ef2b0:   481ef558 48fda4e8 481ef2e8 49c18760

0x481ef2c0:   48fda4e8 6dacc260 481ef558 48fdd428

0x481ef2d0:   490308e4 490308e4 481ef558 49c18760

0x481ef2e0:   49c18720 49c18760 48fda4e8 6dacc48a

0x481ef2f0:   00000001 481ef32c 48fdd428 481ef558

0x481ef300:   4a0afbb8 00000000 6dacfa74 481ef558

0x481ef310:   481ef32c 0000002a 481ef9c8 00000003

0x481ef320:   00000000 00000004 47f09b50 47f09b50 

 

Instructions: (pc=0x6dac8e7e)

0x6dac8e6e:   f8 77 37 8b f7 0f af f1 3b f0 77 2e 8b c6 33 d2

0x6dac8e7e:   f7 f1 3b f8 75 24 8b 44 24 18 8b 15 3c 6e bf 6d 

 

 

Stack: [0x481a0000,0x481f0000),  sp=0x481ef2b0,  free space=316k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code)

V  [jvm.dll+0x1e8e7e]

 

 

Current CompileTask:

opto:875
org.apache.derby.impl.services.locks.LockControl.isGrantable(ZLjava/lang
/Object;Ljava/lang/Object;)Z (158 bytes)

 

--- CRASH3 ---

#

# An unexpected error has been detected by HotSpot Virtual Machine:

#

#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d9796d8, pid=1124,
tid=1192

#

# Java VM: Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode)

# Problematic frame:

# V  [jvm.dll+0x996d8]

#

 

---------------  T H R E A D  ---------------

 

Current thread (0x47f0a468):  JavaThread "CompilerThread1" daemon
[_thread_in_native, id=1192]

 

siginfo: ExceptionCode=0xc0000005, reading address 0x10200694

 

Registers:

EAX=0x102000c0, EBX=0x00000100, ECX=0x4899fe60, EDX=0x00020000

ESP=0x4823ed00, EBP=0x00004000, ESI=0x4899f878, EDI=0x00200000

EIP=0x6d9796d8, EFLAGS=0x00010206

 

Top of Stack: (sp=0x4823ed00)

0x4823ed00:   00000004 4a2376c4 00000000 4823f49c

0x4823ed10:   6dadf3c5 4a2376e8 6dadf415 00000010

0x4823ed20:   4a2376c4 4899f878 4a2376c4 00000000

0x4823ed30:   4823f49c 00000000 00000004 4899f888

0x4823ed40:   00000000 00000004 6dadf3c5 4a2376c4

0x4823ed50:   4899f878 00000000 00000001 4823f49c

0x4823ed60:   4a237750 4823f3a0 00000000 00000000

0x4823ed70:   00000002 4823edac 00000000 00000003 

 

Instructions: (pc=0x6d9796d8)

0x6d9796c8:   3f 85 b9 d4 05 00 00 74 37 8b 46 8c 85 c0 74 30

0x6d9796d8:   85 98 d4 05 00 00 74 28 8b 90 b0 00 00 00 8b 81 

 

 

Stack: [0x481f0000,0x48240000),  sp=0x4823ed00,  free space=315k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code)

V  [jvm.dll+0x996d8]

 

[error occurred during error reporting, step 120, id 0xc0000005]

 

 

Current CompileTask:

opto:2864
org.apache.derby.impl.store.raw.data.FileContainer.trackUnfilledPage(JZ)
V (20 bytes)

 

--- CRASH4 ---

#

# An unexpected error has been detected by HotSpot Virtual Machine:

#

#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00bb906a, pid=1316,
tid=480

#

# Java VM: Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode)

# Problematic frame:

# J  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_264()Z

#

 

---------------  T H R E A D  ---------------

 

Current thread (0x48c75a08):  JavaThread "JobCreationThread" daemon
[_thread_in_Java, id=480]

 

siginfo: ExceptionCode=0xc0000005, reading address 0xffffffff

 

Registers:

EAX=0x4468fdb8, EBX=0x4c90ef40, ECX=0x0af77408, EDX=0x0af77408

ESP=0x4c90ef0d, EBP=0x4c90ef64, ESI=0x5a8b7c49, EDI=0x4c90ef70

EIP=0x00bb906a, EFLAGS=0x00010202

 

Top of Stack: (sp=0x4c90ef0d)

0x4c90ef0d:   1c0af774 3c4c90ef 6b4c90ef 0800972b

0x4c90ef1d:   be0af774 b1446904 444468ff 004c90ef

0x4c90ef2d:   64009a60 704c90ef 444c90ef 404c90ef

0x4c90ef3d:   6b4c90ef 0800972b 480af774 be4c90ef

0x4c90ef4d:   70446904 004c90ef 00446db8 c8000000

0x4c90ef5d:   70446904 904c90ef 6b4c90ef 1800972b

0x4c90ef6d:   0803f466 740af774 c94c90ef 9844690e

0x4c90ef7d:   004c90ef 00446db8 d8000000 9844690e 

 

Instructions: (pc=0x00bb906a)

0x00bb905a:   74 24 18 89 4c 24 04 8b d1 1f be 49 7c 8b 5a 44

0x00bb906a:   8b 72 40 3b f3 74 1e 8b 5e 0c 89 6c 24 10 89 7c 

 

 

Stack: [0x4c8c0000,0x4c910000),  sp=0x4c90ef0d,  free space=315k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code)

J  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_264()Z

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_227()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_191()Z+6

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_121()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_70()Z+161

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3_23()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_100()Z+43

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_56()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3_9()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_226()Z+72

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_188()Z+18

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_116()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_68()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_67()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3_18()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_115()Z+100

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3R_66()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_3_17()Z+1

j  org.apache.derby.impl.sql.compile.SQLParser.jj_2_17(I)Z+19

j
org.apache.derby.impl.sql.compile.SQLParser.selectList()Lorg/apache/derb
y/impl/sql/compile/ResultColumnList;+90

j
org.apache.derby.impl.sql.compile.SQLParser.querySpecification()Lorg/apa
che/derby/impl/sql/compile/ResultSetNode;+24

j
org.apache.derby.impl.sql.compile.SQLParser.simpleTable()Lorg/apache/der
by/impl/sql/compile/ResultSetNode;+33

j
org.apache.derby.impl.sql.compile.SQLParser.nonJoinQueryPrimary()Lorg/ap
ache/derby/impl/sql/compile/ResultSetNode;+41

j
org.apache.derby.impl.sql.compile.SQLParser.nonJoinQueryTerm(Lorg/apache
/derby/impl/sql/compile/ResultSetNode;I)Lorg/apache/derby/impl/sql/compi
le/ResultSetNode;+1

j
org.apache.derby.impl.sql.compile.SQLParser.queryExpression(Lorg/apache/
derby/impl/sql/compile/ResultSetNode;I)Lorg/apache/derby/impl/sql/compil
e/ResultSetNode;+3

j
org.apache.derby.impl.sql.compile.SQLParser.preparableSelectStatement(Z)
Lorg/apache/derby/impl/sql/compile/CursorNode;+20

j
org.apache.derby.impl.sql.compile.SQLParser.preparableSQLDataStatement()
Lorg/apache/derby/impl/sql/compile/QueryTreeNode;+121

j
org.apache.derby.impl.sql.compile.SQLParser.StatementPart([Lorg/apache/d
erby/impl/sql/compile/Token;)Lorg/apache/derby/impl/sql/compile/QueryTre
eNode;+337

j
org.apache.derby.impl.sql.compile.SQLParser.Statement(Ljava/lang/String;
[Ljava/lang/Object;)Lorg/apache/derby/impl/sql/compile/QueryTreeNode;+8

j
org.apache.derby.impl.sql.compile.ParserImpl.parseGoalProduction(Ljava/l
ang/String;[Ljava/lang/Object;)Lorg/apache/derby/impl/sql/compile/QueryT
reeNode;+11

j
org.apache.derby.impl.sql.compile.ParserImpl.parseStatement(Ljava/lang/S
tring;[Ljava/lang/Object;)Lorg/apache/derby/impl/sql/compile/QueryTreeNo
de;+59

j
org.apache.derby.impl.sql.GenericStatement.prepMinion(Lorg/apache/derby/
iapi/sql/conn/LanguageConnectionContext;Z[Ljava/lang/Object;Lorg/apache/
derby/iapi/sql/dictionary/SchemaDescriptor;Z)Lorg/apache/derby/iapi/sql/
PreparedStatement;+562

j
org.apache.derby.impl.sql.GenericStatement.prepare(Lorg/apache/derby/iap
i/sql/conn/LanguageConnectionContext;Z)Lorg/apache/derby/iapi/sql/Prepar
edStatement;+12

j
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareI
nternalStatement(Lorg/apache/derby/iapi/sql/dictionary/SchemaDescriptor;
Ljava/lang/String;ZZ)Lorg/apache/derby/iapi/sql/PreparedStatement;+15

j
org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Lorg/apache/der
by/impl/jdbc/EmbedConnection;Ljava/lang/String;ZIIII[I[Ljava/lang/String
;)V+67

j
org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Lorg/apache/d
erby/impl/jdbc/EmbedConnection;Ljava/lang/String;ZIIII[I[Ljava/lang/Stri
ng;)V+16

j
org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Lorg/apache/d
erby/impl/jdbc/EmbedConnection;Ljava/lang/String;ZIIII[I[Ljava/lang/Stri
ng;)V+16

j
org.apache.derby.jdbc.Driver30.newEmbedPreparedStatement(Lorg/apache/der
by/impl/jdbc/EmbedConnection;Ljava/lang/String;ZIIII[I[Ljava/lang/String
;)Ljava/sql/PreparedStatement;+19

j
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Ljava/lang/S
tring;IIII[I[Ljava/lang/String;)Ljava/sql/PreparedStatement;+33

j
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Ljava/lang/S
tring;II)Ljava/sql/PreparedStatement;+11

j
org.apache.commons.dbcp.DelegatingConnection.prepareStatement(Ljava/lang
/String;II)Ljava/sql/PreparedStatement;+16

j
org.apache.commons.dbcp.PoolingDriver$PoolGuardConnectionWrapper.prepare
Statement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;+11

j  <clipclip - to avoid info leakage>

v  ~StubRoutines::call_stub

V  [jvm.dll+0xf98c0]

 

--- CRASH5 ---

#

# An unexpected error has been detected by HotSpot Virtual Machine:

#

#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d9796d8, pid=1340,
tid=1428

#

# Java VM: Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode)

# Problematic frame:

# V  [jvm.dll+0x996d8]

#

 

---------------  T H R E A D  ---------------

 

Current thread (0x47f09618):  JavaThread "CompilerThread0" daemon
[_thread_in_native, id=1428]

 

siginfo: ExceptionCode=0xc0000005, reading address 0x028005d4

 

Registers:

EAX=0x02800000, EBX=0x00000100, ECX=0x008b7798, EDX=0x00020000

ESP=0x481eed80, EBP=0x00004000, ESI=0x008b71b0, EDI=0x00200000

EIP=0x6d9796d8, EFLAGS=0x00010206

 

Top of Stack: (sp=0x481eed80)

0x481eed80:   00000004 47fc1804 47fc11e0 481ef51c

0x481eed90:   6dadf3c5 47fc1828 6dadf415 00000010

0x481eeda0:   47fc1804 008b71b0 47fc1804 47fc11e0

0x481eedb0:   481ef51c 00000000 00000004 008b71c0

0x481eedc0:   00000000 00000004 6dadf3c5 47fc1804

0x481eedd0:   008b71b0 47fc11e0 00000001 481ef51c

0x481eede0:   47fc1a5c 481ef420 00000000 00000000

0x481eedf0:   00000002 481eee2c 00000000 00000003 

 

Instructions: (pc=0x6d9796d8)

0x6d9796c8:   3f 85 b9 d4 05 00 00 74 37 8b 46 8c 85 c0 74 30

0x6d9796d8:   85 98 d4 05 00 00 74 28 8b 90 b0 00 00 00 8b 81 

 

 

Stack: [0x481a0000,0x481f0000),  sp=0x481eed80,  free space=315k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code)

V  [jvm.dll+0x996d8]

 

[error occurred during error reporting, step 120, id 0xc0000005]

 

 

Current CompileTask:

opto:  3  !
sun.net.www.ParseUtil.decode(Ljava/lang/String;)Ljava/lang/String; (194
bytes)

 

--- CRASH6 ---

#

# An unexpected error has been detected by HotSpot Virtual Machine:

#

#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d96e463, pid=708,
tid=1048

#

# Java VM: Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode)

# Problematic frame:

# V  [jvm.dll+0x8e463]

#

 

---------------  T H R E A D  ---------------

 

Current thread (0x008cc960):  VMThread [id=1048]

 

siginfo: ExceptionCode=0xc0000005, reading address 0x54c8c758

 

Registers:

EAX=0x54c8c758, EBX=0x44c8bf40, ECX=0x00000003, EDX=0x6dbd0154

ESP=0x47eaf9d4, EBP=0x44c8be90, ESI=0x44c8c368, EDI=0x00000025

EIP=0x6d96e463, EFLAGS=0x00010202

 

Top of Stack: (sp=0x47eaf9d4)

0x47eaf9d4:   43a302f0 6dc32ab8 00000000 0076fda0

0x47eaf9e4:   6dadc831 44c8be90 6dadc7c5 6dc32ab8

0x47eaf9f4:   6dadcb1a 6dc37a98 6db72642 6dc37a98

0x47eafa04:   6dc32ab8 0076fda0 00000000 6d9a570c

0x47eafa14:   6dc32ab8 00000000 00000001 0076fda0

0x47eafa24:   00000000 00000000 6d9a6e65 00000001

0x47eafa34:   00000000 00000001 00000002 6dc32ab8

0x47eafa44:   6dc32ab8 00000001 0076fda0 0076fda0 

 

Instructions: (pc=0x6d96e463)

0x6d96e453:   74 08 3c 66 74 04 3c 08 75 19 8b 03 85 c0 74 13

0x6d96e463:   8b 08 83 e1 03 80 f9 03 74 09 53 e8 6d e2 16 00 

 

 

Stack: [0x47e60000,0x47eb0000),  sp=0x47eaf9d4,  free space=318k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
C=native code)

V  [jvm.dll+0x8e463]

 

VM_Operation (0x4aa5e99c): full generation collection, mode: safepoint,
requested by thread 0x48f01618

 

 


Mime
View raw message