db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kathey Marsden (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-4239) corruption on z/OS with storerecovery oc_rec? tests. ERROR XSLA7: Cannot redo operation null in the log.
Date Sun, 31 May 2009 14:39:07 GMT

    [ https://issues.apache.org/jira/browse/DERBY-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714868#action_12714868
] 

Kathey Marsden commented on DERBY-4239:
---------------------------------------

I was doing some more runs on Windows with the patch because I accidentally deleted the database/log
with the EOFException and I got a hang.   I don't really understand what is getting hung up
though.
This is with IBM 1.6 and ReproCorruptionBackgroundCheckpoint.

2XMFULLTHDDUMP Full thread dump J9 VM (J2RE 6.0 IBM J9 2.4 Windows XP x86-32 build jvmwi3260sr5-20090516_3558820090516_035588_lHdSMr,
native threads):
3XMTHREADINFO      "JIT Compilation Thread" TID:0x41CB1900, j9thread_t:0x00292630, state:CW,
prio=10
3XMTHREADINFO1            (native thread ID:0x1028, native priority:0xB, native policy:UNKNOWN)
3XMTHREADINFO      "Gc Slave Thread" TID:0x4209C500, j9thread_t:0x00292FC0, state:CW, prio=5
3XMTHREADINFO1            (native thread ID:0x1638, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO      "derby.antiGC" TID:0x41ED2100, j9thread_t:0x4276259C, state:CW, prio=1
3XMTHREADINFO1            (native thread ID:0x970, native priority:0x1, native policy:UNKNOWN)
4XESTACKTRACE          at java/lang/Object.wait(Native Method)
4XESTACKTRACE          at java/lang/Object.wait(Object.java:167)
4XESTACKTRACE          at org/apache/derby/impl/services/monitor/AntiGC.run(Bytecode PC:15)
4XESTACKTRACE          at java/lang/Thread.run(Thread.java:735)
3XMTHREADINFO      "Thread-2" TID:0x42934500, j9thread_t:0x42762A64, state:CW, prio=5
3XMTHREADINFO1            (native thread ID:0x13E8, native priority:0x5, native policy:UNKNOWN)
4XESTACKTRACE          at java/lang/Object.wait(Native Method)
4XESTACKTRACE          at java/lang/Object.wait(Object.java:167)
4XESTACKTRACE          at java/util/Timer$TimerImpl.run(Timer.java:221)
3XMTHREADINFO      "derby.rawStoreDaemon" TID:0x42933300, j9thread_t:0x42762F2C, state:CW,
prio=5
3XMTHREADINFO1            (native thread ID:0xF30, native priority:0x5, native policy:UNKNOWN)
4XESTACKTRACE          at java/lang/Object.wait(Native Method)
4XESTACKTRACE          at java/lang/Object.wait(Object.java:196(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/services/daemon/BasicDaemon.rest(Bytecode
PC:3(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/services/daemon/BasicDaemon.run(Bytecode PC:22)
4XESTACKTRACE          at java/lang/Thread.run(Thread.java:735)
3XMTHREADINFO      "Thread-6" TID:0x42933900, j9thread_t:0x42763190, state:CW, prio=5
3XMTHREADINFO1            (native thread ID:0x143C, native priority:0x5, native policy:UNKNOWN)
4XESTACKTRACE          at sun/nio/ch/FileChannelImpl.force0(Native Method)
4XESTACKTRACE          at sun/nio/ch/FileChannelImpl.force(FileChannelImpl.java:364(Compiled
Code))
4XESTACKTRACE          at org/apache/derby/impl/io/DirRandomAccessFile4.sync(Bytecode PC:77(Compiled
Code))
4XESTACKTRACE          at org/apache/derby/impl/store/raw/log/LogToFile.syncFile(Bytecode
PC:77(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/store/raw/log/LogToFile.writeControlFile(Bytecode
PC:394(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/store/raw/log/LogToFile.checkpointWithTran(Bytecode
PC:341(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/store/raw/log/LogToFile.checkpoint(Bytecode
PC:16(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/store/raw/RawStore.checkpoint(Bytecode PC:16(Compiled
Code))
4XESTACKTRACE          at org/apache/derby/impl/store/access/RAMAccessManager.checkpoint(Bytecode
PC:16(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/db/BasicDatabase.checkpoint(Bytecode PC:16(Compiled
Code))
4XESTACKTRACE          at org/apache/derby/catalog/SystemProcedures.SYSCS_CHECKPOINT_DATABASE(Bytecode
PC:16(Compiled Code))
4XESTACKTRACE          at org/apache/derby/exe/acd381409ax0121x944ex95fex00000008ed900.g0(Bytecode
PC:16(Compiled Code))
4XESTACKTRACE          at sun/reflect/GeneratedMethodAccessor3.invoke(Bytecode PC:16(Compiled
Code))
4XESTACKTRACE          at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java(Compiled
Code))
4XESTACKTRACE          at java/lang/reflect/Method.invoke(Method.java:578(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/services/reflect/ReflectMethod.invoke(Bytecode
PC:6(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/sql/execute/CallStatementResultSet.open(Bytecode
PC:6(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/sql/GenericPreparedStatement.executeStmt(Bytecode
PC:6(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/sql/GenericPreparedStatement.execute(Bytecode
PC:4(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/jdbc/EmbedStatement.executeStatement(Bytecode
PC:4(Compiled Code))
4XESTACKTRACE          at org/apache/derby/impl/jdbc/EmbedStatement.execute(Bytecode PC:152(Compiled
Code))
4XESTACKTRACE          at org/apache/derby/impl/jdbc/EmbedStatement.execute(Bytecode PC:7(Compiled
Code))
4XESTACKTRACE          at ReproCorruptionBackgroundCheckpoint$1.run(ReproCorruptionBackgroundCheckpoint.java:97(Compiled
Code))
3XMTHREADINFO      "DestroyJavaVM helper thread" TID:0x41CB1300, j9thread_t:0x002923CC, state:CW,
prio=5
3XMTHREADINFO1            (native thread ID:0x808, native priority:0x5, native policy:UNKNOWN)
NULL           ------------------------------------------------------------------------

> corruption on z/OS with storerecovery oc_rec? tests.  ERROR XSLA7: Cannot redo operation
null in the log.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: DERBY-4239
>                 URL: https://issues.apache.org/jira/browse/DERBY-4239
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.1.3.3, 10.2.2.1, 10.3.2.1, 10.4.2.0, 10.5.1.1, 10.6.0.0
>         Environment: z/OS z10 processor. 
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pmz3160sr4-20090219_01(SR4))
> IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 z/OS s390-31 jvmmz3160-20090215_29883 (JIT
enabled, AOT enabled)
> J9VM - 20090215_029883_bHdSMr
> JIT  - r9_20090213_2028
> GC   - 20090213_AA)
> JCL  - 20090218_01
> also 
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pmz3160sr2ifix-20081021_01(SR2+IZ32776+IZ33456))
> IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 z/OS s390-31 jvmmz3160ifx-20081010_24288
(JIT enabled, AOT enabled)
> J9VM - 20081009_024288_bHdSMr
> JIT  - r9_20080721_1330ifx2
> GC   - 20080724_AA)
> JCL  - 20080808_02
>            Reporter: Kathey Marsden
>            Assignee: Mike Matrigali
>            Priority: Critical
>         Attachments: badlogsizes.txt, derby-4239_1.diff, DERBY-4239_2.diff, DERBY-4239_3.diff,
derby.log, derby.log, derby_dumponly.zip, goodlogsizes.txt, identifyBadContainer.ksh, reproBackgroundCheckpoint.zip,
reproDerby4239.zip, wombat_keeplog_notcorrupt.zip, wombat_with_keeplog.zip
>
>
> I saw corruption on z/OS with the storerecovery tests and 10.5.1.1.  The failure comes
in oc_rec3 trying to connect to the database, but the actual problem seems to have occurred
with the prior test oc_rec2.  The problem is somewhat intermittent, happening approximately
1/4 times.  I extracted the case from the harness and will attach the reproduction and run
the script repro.ksh.  The script will loop up to 50 times until it gets the failure which
looks like.
> ERROR XSLA7: Cannot redo operation null in the log.
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.impl.store.raw.log.FileLogger.redo(Unknown Source)
> 	at org.apache.derby.impl.store.raw.log.LogToFile.recover(Unknown Source)
> 	at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
> 	at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
> 	at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown
Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown
Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
> 	at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
> 	at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> 	at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
> 	at java.sql.DriverManager.getConnection(DriverManager.java:311)
> 	at java.sql.DriverManager.getConnection(DriverManager.java:268)
> 	at CheckTables.main(CheckTables.java:8)
> Caused by: ERROR XSDBB: Unknown page format at page Page(16,Container(0, 1073)), page
dump follows: Hex dump:
> 00000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> 00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> <snip lots of 000's>
> I ran it with 10.3 and it completed all 50 iterations, so whether JVM or Derby issue
it seems new since 10.3. (I haven't tried with 10.4).  Oddly I have run tests many times before
on this machine using in the 10.5.1.1 release and the same jvm and have never seen this failure,
so am looking into whether maybe something changed on the machine or environment.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message