Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Message-ID: <1367470694.1214567985102.JavaMail.jira@brutus>
Date: Fri, 27 Jun 2008 04:59:45 -0700 (PDT)
From: "Knut Anders Hatlen (JIRA)" <jira@apache.org>
To: derby-dev@db.apache.org
Subject: [jira] Commented: (DERBY-3732) SQL Length function materializes lob
 into memory
In-Reply-To: <953379663.1213917585016.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/DERBY-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608741#action_12608741 ] 

Knut Anders Hatlen commented on DERBY-3732:
-------------------------------------------

The setting of derby.storage.pageCacheSize will only be respected if the database is booted by that test. I see it is set to 100 when it's run as a separate test, but not when it's run as part of a larger suite.

It would be good to document in a comment why the optimizer override is needed.

I was going to suggest that we should call skip(Long.MAX_VALUE) instead of skip(Integer.MAX_VALUE), but then I saw that it would probably trigger a bug in ArrayInputStream.skip() (will log a separate bug for that).

Tiny nit: testBlobLength(boolean) should be private so that it's clearer that it's not a top-level test case.

> SQL Length function materializes lob into memory
> ------------------------------------------------
>
>                 Key: DERBY-3732
>                 URL: https://issues.apache.org/jira/browse/DERBY-3732
>             Project: Derby
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 10.3.3.0, 10.4.1.3, 10.5.0.0
>            Reporter: Kathey Marsden
>            Priority: Minor
>         Attachments: derby-3732_diff.txt, derby-3732_proto_diff.txt, derby-3732_skip2_diff.txt, derby-3732_skip_diff.txt, LengthLargeLob.zip, LengthThruBlob.java
>
>
> Currently the SQL length function materializes the entire lob into memory. In SQLBinary.getLength() we have 
> public final int	getLength() throws StandardException
> 	{
> 		if (stream != null) {
> 			if (streamValueLength != -1)
> 				return streamValueLength;
> 		}
> 		return (getBytes() == null) ? 0 : getBytes().length;
> 	}
> Which actually is doubly bad because we call getBytes twice and materialize it twice.
> It would be good to read the length from the stream if available and otherwise stream the value to get the length, rather than materializing it into memory.
> To reproduce, run the attached repro.
> java -Xmx16M  LengthLargeLob
> It gives an out of memory exception
> Caused by: java.lang.OutOfMemoryError: Java heap space
>         at org.apache.derby.iapi.types.SQLBinary.readFromStream(SQLBinary.java:415)
>         at org.apache.derby.iapi.types.SQLBinary.readExternal(SQLBinary.java:318)
>         at org.apache.derby.iapi.types.SQLBinary.getValue(SQLBinary.java:220)
>         at org.apache.derby.iapi.types.SQLBinary.getBytes(SQLBinary.java:210)
>         at org.apache.derby.iapi.types.SQLBinary.getLength(SQLBinary.java:250)
>         at org.apache.derby.impl.sql.execute.BaseActivation.getDB2Length(BaseActivation.java:1684)
>         at org.apache.derby.exe.acf81e0010x011axa317x5db8x0000003d9dc81.e1(Unknown Source)
>         at org.apache.derby.impl.services.reflect.DirectCall.invoke(ReflectGeneratedClass.java:141)
>         at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(ProjectRestrictResultSet.java:497)
>         at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:291)
>         at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
>         at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
>         ... 2 more
> [
>  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.