db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DERBY-6309) Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while importing data
Date Fri, 30 Aug 2013 08:43:51 GMT

    [ https://issues.apache.org/jira/browse/DERBY-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13754502#comment-13754502
] 

Knut Anders Hatlen commented on DERBY-6309:
-------------------------------------------

Thank you!

I think the Node class is most likely used for sorting values that are to be inserted into
an index. Could it be that there is a PRIMARY KEY or UNIQUE constraint on one or more of the
columns in the table? Derby uses indexes to enforce those constraints.

The problem might be related to DERBY-6111, where an OOME was seen when creating an index
on a large table. But that report said it was the page cache that filled up the heap, not
the sort buffer, so it could also be something completely different.
                
> Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while importing data
> --------------------------------------------------------------------------------------
>
>                 Key: DERBY-6309
>                 URL: https://issues.apache.org/jira/browse/DERBY-6309
>             Project: Derby
>          Issue Type: Bug
>    Affects Versions: 10.10.1.1
>            Reporter: Manikandan Chellappan
>         Attachments: DerbyLeakSuspects.jpg
>
>
> While importing large files (> 500 MB) using SYSCS_UTIL.SYSCS_IMPORT_DATA procedure,
derby runs out of heap space. The java process which is running the derby has 2GB of heap
space.  The following exception is logged in the derby.log.
> ERROR 38000: The exception 'java.lang.OutOfMemoryError: Java heap space' was thrown while
evaluating an expression.
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
Source)
> 	at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
> 	at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> 	at org.apache.derby.impl.load.Import.importData(Unknown Source)
> 	at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
> 	at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
> 	at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> 	at oracle.retail.stores.iddi.IDDIDerbyWriter.importTable(IDDIDerbyWriter.java:233)
> 	at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:488)
> 	at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:1)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 	at org.apache.derby.impl.load.ImportReadData.readNextDelimitedRow(Unknown Source)
> 	at org.apache.derby.impl.load.ImportReadData.readNextRow(Unknown Source)
> 	at org.apache.derby.impl.load.ImportAbstract.next(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
Source)
> 	at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
> 	at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> 	at org.apache.derby.impl.load.Import.importData(Unknown Source)
> 	at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
> 	at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
> 	at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message