db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kathey Marsden (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-3732) SQL Length function materializes lob into memory
Date Fri, 20 Jun 2008 19:31:46 GMT

    [ https://issues.apache.org/jira/browse/DERBY-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606869#action_12606869
] 

Kathey Marsden commented on DERBY-3732:
---------------------------------------

Looking at a possible quick fix approach as opposed to reworking the type code.

If I insert the lob using length, I should have a streamValueLength available.  When we call
getLength() stream is set but streamValueLength is not.  Below is the trace when the stream
is set.   It seems to me readRecordFromArray should be able to tell the length and pass that
to setStream().  Does that sound reasonable?

	SQLBlob(SQLBinary).setStream(InputStream) line: 563	
	StoredPage.readRecordFromArray(Object[], int, int[], int[], ArrayInputStream, StoredRecordHeader,
RecordHandle) line: 5592	
	StoredPage.restoreRecordFromSlot(int, Object[], FetchDescriptor, RecordHandle, StoredRecordHeader,
boolean) line: 1497	
	StoredPage(BasePage).fetchFromSlot(RecordHandle, int, Object[], FetchDescriptor, boolean)
line: 459	
	HeapScan(GenericScanController).fetchRows(DataValueDescriptor[][], RowLocation[], BackingStoreHashtable,
long, int[]) line: 759	
	HeapScan.fetchNextGroup(DataValueDescriptor[][], RowLocation[]) line: 324	
	BulkTableScanResultSet.reloadArray() line: 327	
	BulkTableScanResultSet.getNextRowCore() line: 282	
	ProjectRestrictResultSet.getNextRowCore() line: 255	
	ProjectRestrictResultSet(BasicNoPutResultSetImpl).getNextRow() line: 460	
	EmbedResultSet40(EmbedResultSet).movePosition(int, int, String) line: 423	
	EmbedResultSet40(EmbedResultSet).next() line: 367	
	LengthLargeLob.main(String[]) line: 21	


> SQL Length function materializes lob into memory
> ------------------------------------------------
>
>                 Key: DERBY-3732
>                 URL: https://issues.apache.org/jira/browse/DERBY-3732
>             Project: Derby
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 10.3.3.0, 10.4.1.3, 10.5.0.0
>            Reporter: Kathey Marsden
>            Priority: Minor
>         Attachments: LengthLargeLob.zip, LengthThruBlob.java
>
>
> Currently the SQL length function materializes the entire lob into memory. In SQLBinary.getLength()
we have 
> public final int	getLength() throws StandardException
> 	{
> 		if (stream != null) {
> 			if (streamValueLength != -1)
> 				return streamValueLength;
> 		}
> 		return (getBytes() == null) ? 0 : getBytes().length;
> 	}
> Which actually is doubly bad because we call getBytes twice and materialize it twice.
> It would be good to read the length from the stream if available and otherwise stream
the value to get the length, rather than materializing it into memory.
> To reproduce, run the attached repro.
> java -Xmx16M  LengthLargeLob
> It gives an out of memory exception
> Caused by: java.lang.OutOfMemoryError: Java heap space
>         at org.apache.derby.iapi.types.SQLBinary.readFromStream(SQLBinary.java:415)
>         at org.apache.derby.iapi.types.SQLBinary.readExternal(SQLBinary.java:318)
>         at org.apache.derby.iapi.types.SQLBinary.getValue(SQLBinary.java:220)
>         at org.apache.derby.iapi.types.SQLBinary.getBytes(SQLBinary.java:210)
>         at org.apache.derby.iapi.types.SQLBinary.getLength(SQLBinary.java:250)
>         at org.apache.derby.impl.sql.execute.BaseActivation.getDB2Length(BaseActivation.java:1684)
>         at org.apache.derby.exe.acf81e0010x011axa317x5db8x0000003d9dc81.e1(Unknown Source)
>         at org.apache.derby.impl.services.reflect.DirectCall.invoke(ReflectGeneratedClass.java:141)
>         at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(ProjectRestrictResultSet.java:497)
>         at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:291)
>         at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
>         at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
>         ... 2 more
> [
>  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message