db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: Negative test case (Re: [jira] Commented: (DERBY-326) Improve streaming of large objects for network server and client)
Date Wed, 01 Feb 2006 18:31:00 GMT
I would have to do some research to tell exactly what happens in the
store, for read uncommitted.  I believe there is no problem for other
isolation levels as we either keep locks, or copy the data on select.
Deletes just mark it deleted, data doesn't go away until commit.  And
updates of multi-page blobs are not in place so even the same thread
updating the same blob it has opened in another thread is probably all
right.  But all interesting test cases.

Note that I think it is impossible to guarantee an error
on the the next read.  There are MANY possible layers of caching going
on and they depend on the following factors:

o embedded vs. client
o size of the lob vs. page size of the database (blobs less than a page
  are going to be read into memory so deletes/updates won't affect).
o where in the stream you are positioned (from store, multi-page blobs
  are read into memory a page at a time, so reads don't go back to
disk/cache until they get to the next page worth of the blob)
o whether the query had to materialize the blob for some reason (we
  try hard not to, but there are cases that have to).
o activity of background space reclamation threads (currently deleted
rows are not reclaimed until all rows on the page are deleted, so
deleting a row with a blob may not remove the blob -- there are some
reported derby issues to increase the reclamation for rows with blobs
in them)

An interesting store centric test would create a table with big rows
such that there is one row per page (one way is to add a bunch of char
fields such that rows length is bigger than a page) and add a blob column.
create a blob that is bigger than the page, say something like 128k.
Then in embedded mode read the first byte as a stream.  A simple select
of the row by primary key and fetch of the column as a stream should
let derby stream the blob out rather than materialize it.
Then as bryan suggests delete and commit the delete in another connection.
Then sleep for awhile to allow background reclamation of the deleted row.
And finally stream the rest of blob out.

I don't know the details of what kind of streaming is going on in the
network layer in blob streaming.


TomohitoNakayama wrote:

> Hello Kathey.
> 
> Kathey Marsden (JIRA) wrote:
> 
>> In the email:
>> http://mail-archives.apache.org/mod_mbox/db-derby-dev/200601.mbox/%3c43D01EA1.2000708@amberpoint.com%3e
>>
>> Bryan asked
>> What sort of faults are we likely to hit? Are things
>> like "corrupt page in database" the most likely?
>>
>> I don't know what is most likely, but I did think of one test case
>> likely to cause an IOException.
>> If the reader is in TRANSACTION_READ_UNCOMMITTED isolation mode and
>> then another connection updated the LOB,  the reader  should get an
>> IOException. on the next read.
>>
> Reading your comment above, I wrote test code attached to this mail, and
> tried executing it with my latest patch, which is not submitted yet.
> Then I found no Exception happens at all.
> 
> Are there any misunderstanding you in my test code ?
> Or is this unlucky success ?
> 
> Best regards.
> 
> 
> ------------------------------------------------------------------------
> 
> /*
> 
> Derby - Class org.apache.derbyTesting.functionTests.tests.lang.DirtyReadOfLob
> 
> Copyright 1999, 2005 The Apache Software Foundation or its licensors, as applicable.
> 
> Licensed under the Apache License, Version 2.0 (the "License");
> you may not use this file except in compliance with the License.
> You may obtain a copy of the License at
> 
> http://www.apache.org/licenses/LICENSE-2.0
> 
> Unless required by applicable law or agreed to in writing, software
> distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the License for the specific language governing permissions and
> limitations under the License.
> 
> */
> 
> package org.apache.derbyTesting.functionTests.tests.lang;
> 
> import java.io.InputStream;
> import java.io.ByteArrayInputStream;
> import java.sql.Connection;
> import java.sql.Statement;
> import java.sql.PreparedStatement;
> import java.sql.ResultSet;
> 
> import java.io.IOException;
> import java.sql.SQLException;
> 
> import org.apache.derby.tools.ij;
> 
> public class DirtyReadOfLob {
>     
>     
>     public static void main(String[] args){
> 	
> 	try{
> 	    ij.getPropertyArg(args);
> 	    
> 	    
> 	    createTestTable();
> 	    
> 	    Connection conn1 = ij.startJBMS();
> 	    Connection conn2 = ij.startJBMS();
> 	    
> 	    testDirtyReadOfLob( conn1,
> 				conn2 );
> 	
> 	    conn1.close();
> 	    conn2.close();
> 
> 	}catch(Throwable t){
> 	    t.printStackTrace();
> 	    
> 	}
>     }
>     
>     
>     private static void createTestTable() 
> 	throws SQLException,
> 	       IllegalAccessException,
> 	       ClassNotFoundException,
> 	       InstantiationException
>     {
> 
> 	Connection conn = ij.startJBMS();
> 	
> 	Statement createTableSt = conn.createStatement();
> 	createTableSt.execute("create table TEST_TABLE( TEST_COL blob( 512 ))");
> 	createTableSt.close();
> 
> 	PreparedStatement insertLobSt = 
> 	    conn.prepareStatement("insert into TEST_TABLE( TEST_COL ) values(?)");
> 	
> 	insertLobSt.setBinaryStream(1,createOriginalDataInputStream(),512);
> 	insertLobSt.executeUpdate();
> 	
> 	insertLobSt.close();
> 	
> 	conn.commit();
> 	conn.close();
> 
>     }
>     
>     
>     private static void testDirtyReadOfLob(Connection conn1,
> 					   Connection conn2) 
> 	throws SQLException,
> 	       IOException
>     {
> 
> 	conn1.setTransactionIsolation(Connection.TRANSACTION_READ_UNCOMMITTED);
> 	conn2.setTransactionIsolation(Connection.TRANSACTION_READ_UNCOMMITTED);
> 
> 	conn1.setAutoCommit(false);
> 	conn2.setAutoCommit(false);
> 	
> 	PreparedStatement st = conn1.prepareStatement("select TEST_COL from TEST_TABLE");
> 	ResultSet rs = st.executeQuery();
> 	rs.next();
> 	
> 	InputStream is = rs.getBinaryStream(1);
> 	
> 	read256Bytes(is);
> 	
> 	updateLobValue(conn2);
> 	conn2.commit();
> 	
> 	read256Bytes(is);
> 	
> 	is.close();
> 	rs.close();
> 	st.close();
> 	
> 	conn1.commit();
> 	
>     }
>     
>     
>     private static ByteArrayInputStream createOriginalDataInputStream(){
> 
> 	byte[] originalValue = new byte[ 512 ];
> 
> 	for(int i = 0; i < originalValue.length; i ++){
> 	    originalValue[i] = (byte) (i % 256);
> 	}
> 	
> 	return new ByteArrayInputStream(originalValue);
> 
>     }
> 
>     
>     private static ByteArrayInputStream createUpdatedDataInputStream(){
> 
> 	byte[] updatedValue = new byte[ 512 ];
> 
> 	for(int i = 0; i < updatedValue.length; i ++){
> 	    updatedValue[i] = (byte) ( (updatedValue.length - i) % 256 );
> 	}
> 	
> 	return new ByteArrayInputStream(updatedValue);
> 
>     }
>     
> 
>     private static void read256Bytes(InputStream is) throws IOException{
> 	
> 	for(int i = 0;
> 	    i < 256;
> 	    i ++){
> 	    
> 	    System.out.print(is.read());
> 	    System.out.print(',');
> 
> 	}
> 	
> 	System.out.println();
> 
>     }
> 
>     private static void updateLobValue(Connection conn) throws SQLException{
> 	
> 	PreparedStatement updateSt = conn.prepareStatement("update TEST_TABLE set TEST_COL =
?");
> 	updateSt.setBinaryStream(1,createUpdatedDataInputStream(),512);
> 	updateSt.executeUpdate();
> 	updateSt.close();
>     }
>     
> }

Mime
View raw message