db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: question about synchonization in RAFContainer4.java for page 0 - from DERBY-3347
Date Mon, 16 Jun 2008 19:33:03 GMT
Knut Anders Hatlen wrote:
> Mike Matrigali <mikem_app@sbcglobal.net> writes:
> 
>> There are comments/code in this file about getting synchonization on
>> page 0.  I went through the comments in the file and in the derby issue
>> and my understanding is that the problem is caused by the behavior on
>> windows where the use of a channel affects the current position of the
>> file that the channel is based off of.  The case in point is one thread
>> using non-channel code to operate on page 0 while another thread is
>> using channel code.
> 
> I think this code intended to solve a different problem. Consider this
> situation:
> 
> Thread 1 tries to update the borrowed space on page 0. It does the
> following:
> 
>   a) synchronize on the container object
>   b) read AllocPage.MAX_BORROWED_SPACE bytes from page 0 into a byte
>      array
>   c) update the borrowed space portion of the byte array
>   d) write AllocPage.MAX_BORROWED_SPACE bytes back to page 0
>   e) release synchronization on the container object
> 
> At the same time in Thread 2, the page cache decides to clean page 0. If
> that happens after (b) and before (d), all the non-borrowed space in the
> first AllocPage.MAX_BORROWED_SPACE bytes of page 0 will come from the
> old version of the page, whereas the rest of the bytes come from the new
> version of the page. This means that the new version of page is not
> fully written to disk, although the page cache thinks the page is
> "clean".
> 
> If we synchronize on the container object when cleaning the page, the
> above problem goes away, since the cleaning of the page must happen
> before (a) or after (e). In those cases we'll always end up with the new
> version of the page on the disk. Note that the problem is only present
> at page 0, since the cleaning of any other page would not touch the same
> bytes as Thread 1.
> 
>> What I am missing is why the problem is limited to channel code on page
>> 0?  What stops the case of non-channel code operating on page 0 while
>> channel code is operating on another page and affecting the non-channel
>> page 0 operation?
>>
>> I considered doing the work to just get rid of the non-channel code, but
>> small platform environments including foundation 1.0, 1.1 and 1.1.2
>> don't support nio.
> 
> I thought all the non-channel code in RAFContainer was overloaded by
> methods in RAFContainer4 after DERBY-3347. Which code did you find that
> still used the old I/O API when running JDK>=1.4?
> 
This is by all means not a complete list, at this point I am still just 
doing code inspection, so it is hard to figure out what can't be called 
because it is overridden.  I just concentrated on the calls in the code
that used seek and then io do do writes.  So far here are the ones I 
have seen called, just by doing a quick limited set of tests (I also 
verified that RAFContainer4 calls were being made also):

Got to FileContainer.java!getEmbryonicPage
Got to FileContainer.java!writeAtOffset
Got to RAFContainer!padFile()
Got to RAFContainer!writePage()

I have not looked at all the stack traces, but here is one for each of them:


java.lang.Throwable
     at 
org.apache.derby.impl.store.raw.data.FileContainer.writeAtOffset(FileContainer.java:912)
     at 
org.apache.derby.impl.store.raw.data.FileContainer.writeHeader(FileContainer.java:889)
     at 
org.apache.derby.impl.store.raw.data.RAFContainer.writeRAFHeader(RAFContainer.java:702)
     at 
org.apache.derby.impl.store.raw.data.RAFContainer.clean(RAFContainer.java:549)
     at 
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAndUnkeepEntry(ConcurrentCache.java:551)
     at 
org.apache.derby.impl.services.cache.ConcurrentCache.cleanCache(ConcurrentCache.java:509)
     at 
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAll(ConcurrentCache.java:460)
     at 
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.checkpoint(BaseDataFileFactory.java:1203)
     at 
org.apache.derby.impl.store.raw.log.LogToFile.checkpointWithTran(LogToFile.java:1645)
     at 
org.apache.derby.impl.store.raw.log.LogToFile.checkpoint(LogToFile.java:1484)
     at org.apache.derby.impl.store.raw.RawStore.stop(RawStore.java:363)
     at 
org.apache.derby.impl.services.monitor.TopService.stop(TopService.java:405)
     at 
org.apache.derby.impl.services.monitor.TopService.shutdown(TopService.java:349)
     at 
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:237)
     at 
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:203)
     at 
org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:231)
     at 
org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:119)
     at java.sql.DriverManager.getConnection(Unknown Source)
     at java.sql.DriverManager.getConnection(Unknown Source)
     at org.apache.derby.impl.tools.ij.utilMain.cleanupGo(utilMain.java:416)
     at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:249)
     at org.apache.derby.impl.tools.ij.Main.go(Main.java:215)
     at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181)
     at org.apache.derby.impl.tools.ij.Main.main(Main.java:73)
     at org.apache.derby.tools.ij.main(ij.java:59)

     at 
org.apache.derby.impl.store.raw.data.FileContainer.getEmbryonicPage(FileContainer.java:959)
     at 
org.apache.derby.impl.store.raw.data.RAFContainer.writeRAFHeader(RAFContainer.java:697)
     at 
org.apache.derby.impl.store.raw.data.RAFContainer.clean(RAFContainer.java:549)
     at 
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAndUnkeepEntry(ConcurrentCache.java:551)
     at 
org.apache.derby.impl.services.cache.ConcurrentCache.cleanCache(ConcurrentCache.java:509)
     at 
org.apache.derby.impl.services.cache.ConcurrentCache.cleanAll(ConcurrentCache.java:460)
     at 
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.checkpoint(BaseDataFileFactory.java:1203)
     at 
org.apache.derby.impl.store.raw.log.LogToFile.checkpointWithTran(LogToFile.java:1645)
     at 
org.apache.derby.impl.store.raw.log.LogToFile.checkpoint(LogToFile.java:1484)
     at org.apache.derby.impl.store.raw.RawStore.stop(RawStore.java:363)
     at 
org.apache.derby.impl.services.monitor.TopService.stop(TopService.java:405)
     at 
org.apache.derby.impl.services.monitor.TopService.shutdown(TopService.java:349)
     at 
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:237)
     at 
org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:203)
     at 
org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:231)
     at 
org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:119)
     at java.sql.DriverManager.getConnection(Unknown Source)
     at java.sql.DriverManager.getConnection(Unknown Source)
     at org.apache.derby.impl.tools.ij.utilMain.cleanupGo(utilMain.java:416)
     at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:249)
     at org.apache.derby.impl.tools.ij.Main.go(Main.java:215)
     at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181)
     at org.apache.derby.impl.tools.ij.Main.main(Main.java:73)
     at org.apache.derby.tools.ij.main(ij.java:59)

at 
org.apache.derby.impl.store.raw.data.RAFContainer.padFile(RAFContainer.java:449)
at 
org.apache.derby.impl.store.raw.data.RAFContainer.writeRAFHeader(RAFContainer.java:711)
at 
org.apache.derby.impl.store.raw.data.RAFContainer.run(RAFContainer.java:1343)
at java.security.AccessController.doPrivileged(AccessController.java:251)
at 
org.apache.derby.impl.store.raw.data.RAFContainer.createContainer(RAFContainer.java:824)
at 
org.apache.derby.impl.store.raw.data.RAFContainer4.createContainer(RAFContainer4.java:159)
at 
org.apache.derby.impl.store.raw.data.FileContainer.createIdent(FileContainer.java:459)
at 
org.apache.derby.impl.store.raw.data.RAFContainer.createIdentity(RAFContainer.java:149)
at 
org.apache.derby.impl.services.cache.ConcurrentCache.create(ConcurrentCache.java:388)
at 
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(BaseDataFileFactory.java:812)
at org.apache.derby.impl.store.raw.xact.Xact.addContainer(Xact.java:1329)
at org.apache.derby.impl.store.access.heap.Heap.create(Heap.java:274)

java.lang.Throwable
     at 
org.apache.derby.impl.store.raw.data.RAFContainer.writePage(RAFContainer.java:301)
     at 
org.apache.derby.impl.store.raw.data.CachedPage.writePage(CachedPage.java:782)
     at 
org.apache.derby.impl.store.raw.data.CachedPage.createIdentity(CachedPage.java:303)
     at 
org.apache.derby.impl.services.cache.ConcurrentCache.create(ConcurrentCache.java:388)
     at 
org.apache.derby.impl.store.raw.data.FileContainer.initPage(FileContainer.java:2348)
     at 
org.apache.derby.impl.store.raw.data.FileContainer.makeAllocPage(FileContainer.java:2282)
     at 
org.apache.derby.impl.store.raw.data.FileContainer.findAllocPageForAdd(FileContainer.java:2061)
     at 
org.apache.derby.impl.store.raw.data.FileContainer.newPage(FileContainer.java:1622)
     at 
org.apache.derby.impl.store.raw.data.BaseContainer.addPage(BaseContainer.java:314)
     at 
org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(BaseContainerHandle.java:183)
     at 
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(BaseDataFileFactory.java:872)
     at 
org.apache.derby.impl.store.raw.xact.Xact.addContainer(Xact.java:1329)
     at org.apache.derby.impl.store.access.heap.Heap.create(Heap.java:274)
     at 
org.apache.derby.impl.store.access.heap.HeapConglomerateFactory.createConglomerate(HeapConglomerateFactory.java:195)
     at 
org.apache.derby.impl.store.access.RAMTransaction.createConglomerate(RAMTransaction.java:826)
     at 
org.apache.derby.impl.sql.execute.CreateTableConstantAction.executeConstantAction(CreateTableConstantAction.java:201)
     at 
org.apache.derby.impl.sql.execute.MiscResultSet.open(MiscResultSet.java:64)
     at 
org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:384)
     at 
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1235)
     at 
org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:625)
     at 
org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:555)
     at org.apache.derby.impl.tools.ij.ij.executeImmediate(ij.java:329)
     at org.apache.derby.impl.tools.ij.utilMain.doCatch(utilMain.java:508)
     at 
org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(utilMain.java:350)
     at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:248)
     at org.apache.derby.impl.tools.ij.Main.go(Main.java:215)
     at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181)
     at org.apache.derby.impl.tools.ij.Main.main(Main.java:73)
     at org.apache.derby.tools.ij.main(ij.java:59)

Mime
View raw message