db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikandan Chellappan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DERBY-6309) Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while importing data
Date Thu, 29 Aug 2013 06:37:53 GMT

    [ https://issues.apache.org/jira/browse/DERBY-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753370#comment-13753370
] 

Manikandan Chellappan commented on DERBY-6309:
----------------------------------------------

I am using embedded derby and using jdbc to run the import procedure. The java program triggers
the import based on cron expression, generally the second or third import cycle fails.

The tables are much larger than the one used for recreation. The tables don't have triggers,
but have constraints and indexes.

One of the larger tables has following columns:

    ID_ITM                  VARCHAR(14)         NOT NULL,
    LU_UOM_SLS              VARCHAR(20)         NOT NULL,
    ED_CLR                  VARCHAR(20),
    ED_SZ                   VARCHAR(10),
    LU_STYL                 VARCHAR(4),
    ID_STR_RT               VARCHAR(5)          DEFAULT '0',
    QL_HT_PCKG_CNS          DECIMAL(9,2)        DEFAULT 0,
    ID_SPR                  VARCHAR(20)         DEFAULT '0',
    NM_NMB_SRZ_ITM_MDL      VARCHAR(40),
    QL_UOM_WD_PCKG_CNS      DECIMAL(9,2)        DEFAULT 0,
    TY_WST_BLK_SLS          VARCHAR(20),
    CY_MDL_SRZ_ITM          VARCHAR(4),
    QU_CPC_HLD              DECIMAL(9,2)        DEFAULT 0,
    LU_UOM                  VARCHAR(2),
    QW_ITM_PCK              DECIMAL(9,2)        DEFAULT 0,
    TY_UN_DPLY              VARCHAR(20),
    QU_CB_PCK_ITM           DECIMAL(9,2)        DEFAULT 0,
    QL_PCKG_CNS             DECIMAL(9,2)        DEFAULT 0,
    DE_CLR_MF_SRZ_ITM       VARCHAR(250),
    TY_ITM_STK              VARCHAR(20),
    PE_WST_BLK_SLS          DECIMAL(5,2)        DEFAULT 0,
    LU_UOM_PCKG_CNS_DMN     VARCHAR(20),
    LU_SYS_PRMRY_MS         VARCHAR(20),
    DE_SZ_MF_SRZ_ITM        VARCHAR(250),
    QU_UN_PCK_ITM           DECIMAL(9,2)        DEFAULT 0,
    DC_UN_DPLY_ST_UP        DATE,
    LU_WRTY_MF_SRZ_ITM      VARCHAR(20),
    QW_WT_PCKG_CNS          DECIMAL(9,2)        DEFAULT 0,
    DC_UN_DPLY_TK_DWN       DATE,
    DE_FBRC                 VARCHAR(250),
    DP_UN_DPLY              VARCHAR(20),
    LU_CNT_SLS_WT_UN        VARCHAR(20),
    LU_WRTY_STR_SRZ         VARCHAR(20),
    LU_UOM_WT_PCKG_CNS      VARCHAR(20),
    TY_PKP_CT_STK_ITM       VARCHAR(20),
    LU_UOM_SZ_PCKG_CNS      VARCHAR(20),
    DE_SLH                  VARCHAR(250),
    FL_VLD_SRZ_ITM          CHAR(1),
    TY_SRZ_ITM              VARCHAR(40),
    ID_SRZ_ITM_LB           INTEGER,
    CD_SRZ_CPT_TM           VARCHAR(20)         DEFAULT 'Sale',
    FL_SRZ_CRT_EXT          CHAR(1)             DEFAULT '1',
    FA_PRC_UN_STK_ITM       DECIMAL(9,2)        DEFAULT 0,
    FL_DSD_AZN              CHAR(1),
    DI_PRD_SH_LF            DECIMAL(3,0)        DEFAULT 0,
    DI_LF_SH                DECIMAL(3,0)        DEFAULT 0,
    ID_BRKR                 INTEGER             DEFAULT 0,
    DC_AVLB_FR_SLS          DATE,
    TY_ITM_STPL_PRSH        VARCHAR(20),
    TY_ENV_STK_ITM          VARCHAR(20),
    NM_LCN_ASL              VARCHAR(250),
    TY_SCTY_RQ              VARCHAR(20),
    NM_LCN_SH               VARCHAR(250),
    NM_LCN_SID              VARCHAR(250),
    TY_MTR_HZ_STK_ITM       VARCHAR(20),
    QU_FCG                  DECIMAL(9,2)        DEFAULT 0,
    CP_UN_SL_LS_RCV_BS      DECIMAL(7,3)        DEFAULT 0,
    CP_CST_NT_LS_RCV        DECIMAL(7,3)        DEFAULT 0,
    CP_UN_SL_LND            DECIMAL(7,3)        DEFAULT 0,
    DC_CST_EST_LS_RCV       DATE,
    FL_SHRK_SH_ITM          CHAR(1),
    FL_SWL_SH_ITM           CHAR(1),
    FL_RQ_UN_PRC            CHAR(1),
    FL_FE_RSTK              CHAR(1),
    TS_CRT_RCRD             TIMESTAMP,
    TS_MDF_RCRD             TIMESTAMP          


                
> Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while importing data
> --------------------------------------------------------------------------------------
>
>                 Key: DERBY-6309
>                 URL: https://issues.apache.org/jira/browse/DERBY-6309
>             Project: Derby
>          Issue Type: Bug
>    Affects Versions: 10.10.1.1
>            Reporter: Manikandan Chellappan
>
> While importing large files (> 500 MB) using SYSCS_UTIL.SYSCS_IMPORT_DATA procedure,
derby runs out of heap space. The java process which is running the derby has 2GB of heap
space.  The following exception is logged in the derby.log.
> ERROR 38000: The exception 'java.lang.OutOfMemoryError: Java heap space' was thrown while
evaluating an expression.
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
Source)
> 	at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
> 	at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> 	at org.apache.derby.impl.load.Import.importData(Unknown Source)
> 	at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
> 	at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
> 	at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
> 	at oracle.retail.stores.iddi.IDDIDerbyWriter.importTable(IDDIDerbyWriter.java:233)
> 	at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:488)
> 	at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:1)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 	at org.apache.derby.impl.load.ImportReadData.readNextDelimitedRow(Unknown Source)
> 	at org.apache.derby.impl.load.ImportReadData.readNextRow(Unknown Source)
> 	at org.apache.derby.impl.load.ImportAbstract.next(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
Source)
> 	at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
> 	at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> 	at org.apache.derby.impl.load.Import.importData(Unknown Source)
> 	at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
> 	at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
> 	at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message