db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Korneliussen <Andreas.Kornelius...@Sun.COM>
Subject Re: conflict detection strategies
Date Fri, 17 Feb 2006 16:12:50 GMT
Mike Matrigali wrote:
> I posted some questions about how the delete/update is done, those
> answers would help me understand better what is needed for a solution.
> 
> I am going to start a separate thread concentrating on RowLocation
> guarantees from store.
> 
That is great.

> Some other answers below.
> 
> Andreas Korneliussen wrote:
> 
> 
>>Mike Matrigali wrote:
>>
>>
>>>There are very few cross thread dependencies not managed by locks
>>>currently.  These things add extra complications to current and
>>>future code.  Also I want to understand clearly the new restrictions
>>>be imposted on the access methods (both current and possible
>>>future).  In the future we would like to do more automatic space
>>>reclamation as part of the zero-admin goal, the ability to do this
>>>in the future internal to access methods is probably affected by
>>>the proposals here.
>>>
>>
>>I think the complexities are there already. It seems very hard to know
>>under exactly which conditions, a RowLocation remains valid.  We have
>>assumed that the RowLocation will remain valid as long as we hold a
>>table intent lock (valid means that it either points to the same row or
>>to a deleted row).
>>
>>That is what we concluded from the previous discusssion about validity
>>of RowLocation. If you in the future need to make code, or there already
>>is code, which breaks this assumption, we would need to know which other
>>mechanisms we should use to either verify that the RowLocation is valid,
>>or to block the system to make it invalid.
> 
> There is already code that breaks this assumption, the in place compress
> table.  It currently is only executed as a call to a system procedure:
> http://db.apache.org/derby/docs/dev/ref/rrefproceduresinplacecompress.html
>
> But the hope was in the future to do the same kind of work that this
> procedure does, internally in background.
> 
> As you have already determined that off-line compress is even more of a
> problem, but it does run under an exclusive table lock.  After it runs
> the container you were connected to does not even exist any more.
>

The online compress seems to require table-exclusive locks when passing 
the defragment or truncate_end argument.
There are two testcases in jdbcapi/ConcurrencyTest which test this 
(testDefragmentDuringScan and testTruncateDuringScan).

The tests first deletes all rows except the first and the last. Then it 
commits. Then it opens a SUR in read-uncommitted mode. Read all records 
into the resultset, and position the cursor to afterlast.
Then it runs the defragment or truncate. The test will hang waiting for 
a lock. Finally, it updates the rows correctly.

Output from the tests (modified the test slightly to get more output):
T1: delete records
T1: commit
T2: Read next Tuple:(0,0,17)
T2: Read next Tuple:(9,9,35)
T3: call SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE
T3: DEFRAGMENT
ERROR 40XL1: A lock could not be obtained within the time requested
.... 
org.apache.derby.impl.store.access.RAMTransaction.openScan(RAMTransaction.java:1582)
         at 
org.apache.derby.iapi.db.OnlineCompress.setup_indexes(OnlineCompress.java:605)
         at 
org.apache.derby.iapi.db.OnlineCompress.defragmentRows(OnlineCompress.java:359)
         at 
org.apache.derby.iapi.db.OnlineCompress.compressTable(OnlineCompress.java:227)
         at 
org.apache.derby.catalog.SystemProcedures.SYSCS_INPLACE_COMPRESS_TABLE(SystemProcedures.java:858)
...
        at 
org.apache.derbyTesting.functionTests.tests.jdbcapi.ConcurrencyTest.testCompressDuringScan(ConcurrencyTest.java:777)
        at 
org.apache.derbyTesting.functionTests.tests.jdbcapi.ConcurrencyTest.testDefragmentDuringScan(ConcurrencyTest.java:712)
...

T3: got expected exception
T1: Read first Tuple:(0,0,17)
T1: updateInt(2, 3);
T1: updateRow()
T1: Read last Tuple:(9,9,35)
T1: updateInt(2, 3);
T1: updateRow()
T1: commit
T4: select * from table
T4: Read next Tuple:(0,3,17)
T4: Read next Tuple:(9,3,35)

So to me it seems that our assumptions are correct.. Only a purge is 
allowed with row-level locking, online compress and online defragment 
seems to be blocked by an open cursor on the table.
Maybe that is not how online compress was intended to be ?

Andreas


> 
>>Locks can be used to manage cross thread dependencies, however they are
>>bound to the transaction, and therefore does not help very much for
>>cursors held across commits. So if the only mechanism we can have to
>>ensure that the RowLocations are valid, is by the means of locks, I
>>doubt we can support the feature of scrollable insensitive *holdable*
>>updatable resultset.
> 
> 
> I agree, I don't believe derby is currently architected to correctly
> implement "holdable" SUR.  I don't think the outside event driven
> approach is the right way to go.
> 
>>>It is true that the current access methods don't reuse row locations
>>>until a table level lock is granted.  But your project would be the
>>>first dependency on this outside of the access method implementations
>>>themselves.  It is very clear the contract that the access methods
>>>have with their clients while locks are held on the data they are
>>>looking at, what you are proposing is a contract on unlocked data.
>>>
>>
>>I guess we are the first to use RowLocation without holding a lock on
>>the row. This is necessary, unless we simply make SUR cursors set locks
>>for all rows in the cursor independent from isolation level.
>>
>>
>>>Note that the current "in-place" compress will MOVE rows from one
>>>row location to another if one does not have a row lock on the row.
>>>This is done in the 2nd phase and only holds an intent lock, and
>>>exclusive row locks on the rows being moved.
>>>The off-line compress only does work under an X table lock.
>>>So the row that you are updating actually will exist in the table,
>>>but currently you will request the old location and will get back
>>>a delete row indicator.  I think because of this option 1 does not
>>>work.
>>>
>>
>>Are you saing that RowLocations can be invalidated by "in-place"
>>compress even if we hold a Table intent lock ?
> 
> The "in-place" compress will do the following take a row a row location
> N and do the
> following:
>     insert row at new row location M
>     delete row at old row location N
>     possibly purge old row location
> 
> I don't understand how this will affect your code, it depends how you
> search for the row, and what errors are ok to throw.
> 



Mime
View raw message