db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Thalamati (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-606) SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails on (very) large tables
Date Fri, 01 Dec 2006 23:33:22 GMT
    [ http://issues.apache.org/jira/browse/DERBY-606?page=comments#action_12455046 ] 
            
Suresh Thalamati commented on DERBY-606:
----------------------------------------

Thanks for considering my suggestions.  Extra check(if( !(this instanceof
CompressSpacePageOperation10_2) ) in the  CompressSpacePageOperation.java 
is ok. You need some kind of check in  to correctly call the 
writeExternal/readExternal. Only othe alternative I can think of is to 
check for formatId instead of using the "instanceof". 


While reviewing the latest patches(derby606_impl-v4.diff and derby606_upgrade-v4.diff ), 
noticed you are disabling IN_PLACE_COMPRESS  on soft-upgrade to 10.3, in my 
view that is incorrect. If I understand correctly this bug happens only on 
large data size and does not leave the database in the corrupt state. 

I believe right thing to do is to use the old log record on soft-upgrade to
10.3 and allow compress. If you would like to be real nice to the users 
then you can catch the bugs case and throw an error message, but I think it is
not required; users hitting the bug case first time on a a soft-upgrade is 
almost zero probability. 
  
Index: java/engine/org/apache/derby/impl/store/raw/data/LoggableAllocActions.java

+	if( t.getLogFactory().checkVersion(RawStoreFactory.DERBY_STORE_MAJOR_VERSION_10,
+					RawStoreFactory.DERBY_STORE_MINOR_VERSION_3,
+					"CompressSpace operation on an existing database") )

Above call throws an exception if version is not at the correct level, in this
case if not at  10.3. I think if you pass "null" for feature name , it will
just return true/false. 


Please also fix the upgrade tests also, if you decide to allow compress on soft-upgrade. 

I am sorry for missing this in my previous review. 


There is one major problem with the latest patch, it can make database 
non-recoverable on upgrade from 10.2.

1) Formatid are incorrect for CompressSpacePageOperation10_2 and    CompressSpacePageOperation.

   CompressSpacePageOperation should have the new format id and  the
   CompressSpacePageOperation10_2 should have the OLD one. 

Related changes :

Index: java/engine/org/apache/derby/impl/store/raw/data/CompressSpacePageOperation10_2.java
+
+	/**
+		Return my format identifier.
+	*/
+	public int getTypeFormatId() {
+		return super.getTypeFormatId();
+	}

I am sure , you wanted it to return StoredFormatIds.LOGOP_COMPRESS10_2_SPACE;  

and  also :
Index: java/engine/org/apache/derby/iapi/services/io/RegisteredFormatIds.java
+	/* 465 */   "org.apache.derby.impl.store.raw.data.CompressSpacePageOperation10_2",

That shoud have been
"org.apache.derby.impl.store.raw.data.CompressSpacePageOperation" and you
should change existing entry for compress to "CompressSpacePageOperation10_2".

and the format id numbers should get changed  also :
Index: java/engine/org/apache/derby/iapi/services/io/StoredFormatIds.java
         public static final int LOGOP_COMPRESS_SPACE =
                 (MIN_ID_2 + 454);
 
+	/* org.apache.derby.impl.store.raw.data.CompressSpacePageOperation10_2 */
+        public static final int LOGOP_COMPRESS10_2_SPACE =
+                (MIN_ID_2 + 465);
+



2) In the upgrade test , why you need to insert  104000 rows ? I think you 
can produce the compress log record with less number of records and reduce
the test time.  

+			try {
+				checkDataToCase606(conn, 0, 104000, true);
+			} catch(SQLException sqle) {
+				passed = isExpectedException(sqle, "XSLB1");
+			}



Thanks
-suresh

> SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails on (very) large tables
> --------------------------------------------------------------------
>
>                 Key: DERBY-606
>                 URL: http://issues.apache.org/jira/browse/DERBY-606
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.1.1.0
>         Environment: Java 1.5.0_04 on Windows Server 2003 Web Edition
>            Reporter: Jeffrey Aguilera
>         Assigned To: Mayuresh Nirhali
>             Fix For: 10.3.0.0
>
>         Attachments: A606Test.java, derby606-v2.diff, derby606-v3.diff, derby606_impl-v4.diff,
derby606_upgrade-v4.diff, derby606_v1.diff
>
>
> SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE fails with one of the following error messages
when applied to a very large table (>2GB):
> Log operation null encounters error writing itself out to the log stream, this could
be caused by an errant log operation or internal log buffer full due to excessively large
log operation. SQLSTATE: XJ001: Java exception: ': java.io.IOException'.
> or
> The exception 'java.lang.ArrayIndexOutOfBoundsException' was thrown while evaluating
an expression. SQLSTATE: XJ001: Java exception: ': java.lang.ArrayIndexOutOfBoundsException'.
> In either case, no entry is written to the console log or to derby.log.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message