db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kathey Marsden (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DERBY-5344) updateClob2 test in LobLimitsTest gets OutOfMemoryError on updateRow with embedded
Date Thu, 11 Oct 2012 15:59:03 GMT

     [ https://issues.apache.org/jira/browse/DERBY-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kathey Marsden updated DERBY-5344:
----------------------------------

    Attachment: heapdump.20110720.zip
                heapanalyzer.20110720.215042.24230.0001.jpg

Attaching the heapdump and a screen shot of heap analyzer for the run I did in July. 

The IBM heap analyzer can be found at:
https://www.ibm.com/developerworks/mydeveloperworks/groups/service/html/communityview?communityUuid=4544bafe-c7a2-455f-9d43-eb866ea60091

 The heapdump shows a SQLClob with String (presumably value of the underlying SQLChar) of
size 209.716,216 bytes, so presumably a problem with materialization but the thing I don't
really understand is why this size when the actual lobs in the test are much larger.  I am
thinking probably  something  is happening in EmbedResultSet updateRow()  related to setting
the parameters of the constructed WHERE CURRENT OF statement in this code:
//using quotes around the cursor name to preserve case sensitivity
            updateWhereCurrentOfSQL.append(" WHERE CURRENT OF " + 
                    IdUtil.normalToDelimited(getCursorName()));

            StatementContext currSC = lcc.getStatementContext();
            Activation parentAct = null;

            if (currSC != null) {
                parentAct = currSC.getActivation();
            }

            // Context used for preparing, don't set any timeout (use 0)
            statementContext = lcc.pushStatementContext(isAtomic, false, updateWhereCurrentOfSQL.toString(),
null, false, 0L);

            // A priori, the new statement context inherits the activation of
            // the existing statementContext, so that that activation ends up
            // as parent of the new activation 'act' created below, which will
            // be the activation of the pushed statement context.
            statementContext.setActivation(parentAct);

            org.apache.derby.iapi.sql.PreparedStatement ps = lcc.prepareInternalStatement(updateWhereCurrentOfSQL.toString());
            Activation act = ps.getActivation(lcc, false);

            statementContext.setActivation(act);

            //in this for loop we are assigning values for parameters in sql constructed earlier
with columnname=?,... 
            for (int i=1, paramPosition=0; i<=rd.getColumnCount(); i++) { 
                if (columnGotUpdated[i-1])  //if the column got updated, do following
                    act.getParameterValueSet().getParameterForSet(paramPosition++).setValue(updateRow.getColumn(i));
            }


But that theory may be off as I would have expected that in that case it would be setValue()
that would have been the method that would have run out of memory rather than updateRow().
Maybe setValue got inlined?


Myrna and I also looked at her recent heapdump in heap analyzer.  Hers was quite odd as there
was an orphaned char[] of approximately 39MB (she ran with a smaller heap). I am thinking
maybe in that case the large string was in the process of being garbage collected and so had
no parents but was still there.  Still it shows an inappropriately large string is being built
up.






                
> updateClob2 test in LobLimitsTest gets OutOfMemoryError on updateRow with embedded
> ----------------------------------------------------------------------------------
>
>                 Key: DERBY-5344
>                 URL: https://issues.apache.org/jira/browse/DERBY-5344
>             Project: Derby
>          Issue Type: Bug
>          Components: Store, Test
>    Affects Versions: 10.9.1.0
>            Reporter: Kathey Marsden
>              Labels: derby_triage10_10
>         Attachments: DERBY-5344_dump.jar, heapanalyzer.20110720.215042.24230.0001.jpg,
heapdump.20110720.zip
>
>
> On converting LobLimits.java to LobLimitsTest I noticed this disabled test:
> There is  disabled test in largedata.LobLimitsTest.java which is a carryover from largedata.LobLimits.java
>         // Disabled for now, this will materialize, will open
>         // jira for it.
>         // updateClob2("ClobTest #8.1",conn,selectClob,BIG_LOB_SZ,0,0,10,1,CHARDATAFILE);
> Enabling the test for embedded I noticed it still  can get an out of memory.  (I actually
think I did run it once successfully)
> but on my second full run of the suite I saw it fail.
>  
> java.sql.SQLException: Java exception: ': java.lang.OutOfMemoryError'.
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98)
> 	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:142)
> 	at org.apache.derby.impl.jdbc.Util.javaException(Util.java:299)
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:412)
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:348)
> 	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2290)
> 	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:82)
> 	at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(EmbedResultSet.java:4409)
> 	at org.apache.derby.impl.jdbc.EmbedResultSet.updateRow(EmbedResultSet.java:3788)
> 	at org.apache.derbyTesting.functionTests.tests.largedata.LobLimitsTest.updateClob2(LobLimitsTest.java:1219)
> 	at org.apache.derbyTesting.functionTests.tests.largedata.LobLimitsTest.testClob2(LobLimitsTest.java:304)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> 	at java.lang.reflect.Method.invoke(Method.java:611)
> 	at junit.framework.TestCase.runTest(TestCase.java:154)
> 	at junit.framework.TestCase.runBare(TestCase.java:127)
> 	at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
> 	at junit.framework.TestResult$1.protect(TestResult.java:106)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.framework.TestResult.run(TestResult.java:109)
> 	at junit.framework.TestCase.run(TestCase.java:118)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
> 	at junit.framework.TestSuite.run(TestSuite.java:203)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
> 	at junit.framework.TestSuite.run(TestSuite.java:203)
> 	at junit.textui.TestRunner.doRun(TestRunner.java:116)
> 	at junit.textui.TestRunner.start(TestRunner.java:172)
> 	at junit.textui.TestRunner.main(TestRunner.java:138)
> Caused by: java.sql.SQLException: Java exception: ': java.lang.OutOfMemoryError'.
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122)
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
> 	... 37 more
> Caused by: java.lang.OutOfMemoryError
> 	at org.apache.derby.iapi.services.io.DynamicByteArrayOutputStream.expandBuffer(DynamicByteArrayOutputStream.java:244)
> 	at org.apache.derby.iapi.services.io.DynamicByteArrayOutputStream.write(DynamicByteArrayOutputStream.java:78)
> 	at java.io.DataOutputStream.write(DataOutputStream.java)
> 	at org.apache.derby.iapi.types.SQLChar.writeUTF(SQLChar.java:922)
> 	at org.apache.derby.iapi.types.SQLChar.writeClobUTF(SQLChar.java:960)
> 	at org.apache.derby.iapi.types.SQLClob.writeExternal(SQLClob.java:647)
> 	at org.apache.derby.impl.store.raw.data.StoredPage.logColumn(StoredPage.java:6325)
> 	at org.apache.derby.impl.store.raw.data.StoredPage.logRow(StoredPage.java:4006)
> 	at org.apache.derby.impl.store.raw.data.UpdateOperation.writeOptionalDataToBuffer(UpdateOperation.java:255)
> 	at org.apache.derby.impl.store.raw.data.UpdateOperation.<init>(UpdateOperation.java:106)
> 	at org.apache.derby.impl.store.raw.data.LoggableActions.actionUpdate(LoggableActions.java:80)
> 	at org.apache.derby.impl.store.raw.data.StoredPage.doUpdateAtSlot(StoredPage.java:8602)
> 	at org.apache.derby.impl.store.raw.data.BasePage.updateAtSlot(BasePage.java:1064)
> 	at org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.replace(GenericConglomerateController.java:486)
> 	at org.apache.derby.impl.sql.execute.RowChangerImpl.updateRow(RowChangerImpl.java:523)
> 	at org.apache.derby.impl.sql.execute.UpdateResultSet.collectAffectedRows(UpdateResultSet.java:569)
> 	at org.apache.derby.impl.sql.execute.UpdateResultSet.open(UpdateResultSet.java:264)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:436)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeSubStatement(GenericPreparedStatement.java:306)
> 	at org.apache.derby.impl.jdbc.EmbedResultSet.updateRow(EmbedResultSet.java:3772)
> 	... 29 more
> The test passes on client and in LobLimitsLiteTest  in embedded. To reproduce run largedata.LobLimitsTest,
commenting out the below condition:
>        if (!(usingEmbedded()  && BIGGEST_LOB_SZ  == _2GB)) {
>             updateClob2("ClobTest #8.1",selectClob,BIG_LOB_SZ,0,0,10,CHARDATAFILE);
>         }
>   or likely you can get a smaller standalone test case by adjusting the sizes and adjusting
down the heap with -Xmx

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message