db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: conflict detection strategies
Date Fri, 17 Feb 2006 17:06:45 GMT
It looks like I may have been thinking about future directions vs.
current reality.  The question still is what should the contract
be, rather than what a specific implementation currently provides.



Andreas Korneliussen wrote:
> Mike Matrigali wrote:
> 
>> I posted some questions about how the delete/update is done, those
>> answers would help me understand better what is needed for a solution.
>>
>> I am going to start a separate thread concentrating on RowLocation
>> guarantees from store.
>>
> That is great.
> 
>> Some other answers below.
>>
>> Andreas Korneliussen wrote:
>>
>>
>>> Mike Matrigali wrote:
>>>
>>>
>>>> There are very few cross thread dependencies not managed by locks
>>>> currently.  These things add extra complications to current and
>>>> future code.  Also I want to understand clearly the new restrictions
>>>> be imposted on the access methods (both current and possible
>>>> future).  In the future we would like to do more automatic space
>>>> reclamation as part of the zero-admin goal, the ability to do this
>>>> in the future internal to access methods is probably affected by
>>>> the proposals here.
>>>>
>>>
>>> I think the complexities are there already. It seems very hard to know
>>> under exactly which conditions, a RowLocation remains valid.  We have
>>> assumed that the RowLocation will remain valid as long as we hold a
>>> table intent lock (valid means that it either points to the same row or
>>> to a deleted row).
>>>
>>> That is what we concluded from the previous discusssion about validity
>>> of RowLocation. If you in the future need to make code, or there already
>>> is code, which breaks this assumption, we would need to know which other
>>> mechanisms we should use to either verify that the RowLocation is valid,
>>> or to block the system to make it invalid.
>>
>>
>> There is already code that breaks this assumption, the in place compress
>> table.  It currently is only executed as a call to a system procedure:
>> http://db.apache.org/derby/docs/dev/ref/rrefproceduresinplacecompress.html 
>>
>>
>> But the hope was in the future to do the same kind of work that this
>> procedure does, internally in background.
>>
>> As you have already determined that off-line compress is even more of a
>> problem, but it does run under an exclusive table lock.  After it runs
>> the container you were connected to does not even exist any more.
>>
> 
> The online compress seems to require table-exclusive locks when passing 
> the defragment or truncate_end argument.
> There are two testcases in jdbcapi/ConcurrencyTest which test this 
> (testDefragmentDuringScan and testTruncateDuringScan).
> 
> The tests first deletes all rows except the first and the last. Then it 
> commits. Then it opens a SUR in read-uncommitted mode. Read all records 
> into the resultset, and position the cursor to afterlast.
> Then it runs the defragment or truncate. The test will hang waiting for 
> a lock. Finally, it updates the rows correctly.
> 
> Output from the tests (modified the test slightly to get more output):
> T1: delete records
> T1: commit
> T2: Read next Tuple:(0,0,17)
> T2: Read next Tuple:(9,9,35)
> T3: call SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE
> T3: DEFRAGMENT
> ERROR 40XL1: A lock could not be obtained within the time requested
> .... 
> org.apache.derby.impl.store.access.RAMTransaction.openScan(RAMTransaction.java:1582)

> 
>         at 
> org.apache.derby.iapi.db.OnlineCompress.setup_indexes(OnlineCompress.java:605) 
> 
>         at 
> org.apache.derby.iapi.db.OnlineCompress.defragmentRows(OnlineCompress.java:359) 
> 
>         at 
> org.apache.derby.iapi.db.OnlineCompress.compressTable(OnlineCompress.java:227) 
> 
>         at 
> org.apache.derby.catalog.SystemProcedures.SYSCS_INPLACE_COMPRESS_TABLE(SystemProcedures.java:858)

> 
> ...
>        at 
> org.apache.derbyTesting.functionTests.tests.jdbcapi.ConcurrencyTest.testCompressDuringScan(ConcurrencyTest.java:777)

> 
>        at 
> org.apache.derbyTesting.functionTests.tests.jdbcapi.ConcurrencyTest.testDefragmentDuringScan(ConcurrencyTest.java:712)

> 
> ...
> 
> T3: got expected exception
> T1: Read first Tuple:(0,0,17)
> T1: updateInt(2, 3);
> T1: updateRow()
> T1: Read last Tuple:(9,9,35)
> T1: updateInt(2, 3);
> T1: updateRow()
> T1: commit
> T4: select * from table
> T4: Read next Tuple:(0,3,17)
> T4: Read next Tuple:(9,3,35)
> 
> So to me it seems that our assumptions are correct.. Only a purge is 
> allowed with row-level locking, online compress and online defragment 
> seems to be blocked by an open cursor on the table.
> Maybe that is not how online compress was intended to be ?
> 
> Andreas
> 
> 
>>
>>> Locks can be used to manage cross thread dependencies, however they are
>>> bound to the transaction, and therefore does not help very much for
>>> cursors held across commits. So if the only mechanism we can have to
>>> ensure that the RowLocations are valid, is by the means of locks, I
>>> doubt we can support the feature of scrollable insensitive *holdable*
>>> updatable resultset.
>>
>>
>>
>> I agree, I don't believe derby is currently architected to correctly
>> implement "holdable" SUR.  I don't think the outside event driven
>> approach is the right way to go.
>>
>>>> It is true that the current access methods don't reuse row locations
>>>> until a table level lock is granted.  But your project would be the
>>>> first dependency on this outside of the access method implementations
>>>> themselves.  It is very clear the contract that the access methods
>>>> have with their clients while locks are held on the data they are
>>>> looking at, what you are proposing is a contract on unlocked data.
>>>>
>>>
>>> I guess we are the first to use RowLocation without holding a lock on
>>> the row. This is necessary, unless we simply make SUR cursors set locks
>>> for all rows in the cursor independent from isolation level.
>>>
>>>
>>>> Note that the current "in-place" compress will MOVE rows from one
>>>> row location to another if one does not have a row lock on the row.
>>>> This is done in the 2nd phase and only holds an intent lock, and
>>>> exclusive row locks on the rows being moved.
>>>> The off-line compress only does work under an X table lock.
>>>> So the row that you are updating actually will exist in the table,
>>>> but currently you will request the old location and will get back
>>>> a delete row indicator.  I think because of this option 1 does not
>>>> work.
>>>>
>>>
>>> Are you saing that RowLocations can be invalidated by "in-place"
>>> compress even if we hold a Table intent lock ?
>>
>>
>> The "in-place" compress will do the following take a row a row location
>> N and do the
>> following:
>>     insert row at new row location M
>>     delete row at old row location N
>>     possibly purge old row location
>>
>> I don't understand how this will affect your code, it depends how you
>> search for the row, and what errors are ok to throw.
>>
> 
> 
> 
> 


Mime
View raw message