db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mayuresh Nirhali (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-606) SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails on (very) large tables
Date Wed, 15 Nov 2006 15:40:39 GMT
    [ http://issues.apache.org/jira/browse/DERBY-606?page=comments#action_12450071 ] 
Mayuresh Nirhali commented on DERBY-606:

Thanks Mike and Suresh.

Yes. It makes sense to only change the log record format for this special case.

Looking at possible runtime fix, I feel, this approach would create some Major-Minor version
checks in write/readExternal methods which would not be ideal. Approach of Extending the class
would make the changes much simpler.

So, currently this is what I have in mind. CompressSpacePageOperation will be extended to
form a new class CompressSpacePageOperation10_3. This class over override readExternal, writeExternal
and getTypeFormatId methods. read/writeExternal methods would write the new_highest_page field
without compression and the other fields are compressed and written to the Log. A new ID StoreFormatIds.LOGOP_COMPRESS10_3_SPACE
is created.

Now, when LOggableAllocActions creates CompressSpacePageOperation, a new check will be added
to identify whether this is a soft or hard upgrade. The only way to do this is by calling
LogToFile.checkVersion(Major, Minor). Based on the flag returned either the old CompressSpacePageOperation
or the new one will be called. LogFactory is not accessible at this level in the code and
hence calling checkVersion is not possible. Xact has no interface to get XContext so there
isnt any way to reach LogFactory. I need some inputs here.
While investigating, I also realized that even if you write the record with newHighestPage
set to -1, the log is written successfully but the transaction does not commit because AllocPage.compressSpace
has an assertion (only in DEBUG mode btw) which checks if the newHighestPage is not positive.
Thus an Exception is thrown. I think this assertion check is invalid. Also it causes inconsistency
in the behavior between sane-insane builds as it only happens when SanityManager.DEBUG is
true. Will it be okay to remove this assertion ?

Lastly, If the compress space operation involving data pages spanned across multiple AllocExtents
(which is the current case), the compress operation happens only on the last page. Ideally,
it should clean up the last AllocPage (with referred datapages) and also clean up data pages
which might be empty in the previous AllocationExtent. I realized this when I saw a very small
decrease in physical size when I ran the reproducible testcase. Can someone please confirm
this ?? So, I can file a JIRA for this.

> SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails on (very) large tables
> --------------------------------------------------------------------
>                 Key: DERBY-606
>                 URL: http://issues.apache.org/jira/browse/DERBY-606
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions:
>         Environment: Java 1.5.0_04 on Windows Server 2003 Web Edition
>            Reporter: Jeffrey Aguilera
>         Assigned To: Mayuresh Nirhali
>         Attachments: A606Test.java
> SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails with one of the following error messages
when applied to a very large table (>2GB):
> Log operation null encounters error writing itself out to the log stream, this could
be caused by an errant log operation or internal log buffer full due to excessively large
log operation. SQLSTATE: XJ001: Java exception: ': java.io.IOException'.
> or
> The exception 'java.lang.ArrayIndexOutOfBoundsException' was thrown while evaluating
an expression. SQLSTATE: XJ001: Java exception: ': java.lang.ArrayIndexOutOfBoundsException'.
> In either case, no entry is written to the console log or to derby.log.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message