db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Matrigali (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-3219) Group by query with many aggregate columns and case statements fails with: ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted to read more data than was originally stored
Date Wed, 21 May 2008 19:19:55 GMT

     [ https://issues.apache.org/jira/browse/DERBY-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mike Matrigali updated DERBY-3219:
----------------------------------


I have not used it for a very long time, but there use to be a debug/SANE version only hack
that would allow you to force an external sort - no matter what the memory.  I think
if you set the folllowing, then it will go external as long as you have enough rows to
exceed the buffer max that is also setable.:
derby.debug.true=testSort

See:
opensource/java/testing/org/apache/derbyTesting/functionTests/tests/unit/T_SortController__derby.properties

or
opensource/java/testing/org/apache/derbyTesting/functionTests/tests/store/streamingColumn_derby.properties


The area of trying to auto configure sort into memory or not could use a good idea.  

The question is how to provide a zero admin solution that makes everyone happy,
for a product that both might be used completely embedded as part of a lot of other
components so it probably should not take over all memory of the JVM vs use as
the single db server on the machine at that time so quite desirable to use all 
avaliable resources as efficiently as possible.

The existing code is based on jvm features available many years ago and against assumed default
configurations where 1 megabyte was ALOT of memory (some
of the target devices didn't even have that much memory).
The code really wanted access to MX available memory but the jvm would
not give access to that number.  So it had to work off actual amount of memory free which
may be 0 even if the MX has been set to allow for 100 megabytes more growth.  The current
free number is VERY system dependent - depending on garbage collection, jvm implementation,
OS, scheduling within and outside of the JVM, ...

> Group by query with many aggregate columns and case statements fails with: ERROR XSDA7:
Restore of a serializable or SQLData object of class , attempted to read more data than was
originally stored
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: DERBY-3219
>                 URL: https://issues.apache.org/jira/browse/DERBY-3219
>             Project: Derby
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 10.3.1.4
>            Reporter: Stan Bradbury
>            Assignee: Bryan Pendleton
>         Attachments: maxminPatch.diff, patchWithTest.diff, pivotView.zip, repro.java,
testWithMemControls.diff
>
>
> using the attached database (v10.3) - " select * from pivotview " fails with the stack
trace below.  A view (pivotview_ok) created on a subset of the columns in pivotview executes
fine.  Adding one column back into pivotview_ok causes failures most of the time.  See attached
for view definitions.
> 2007-11-21 00:58:49.421 GMT Thread[main,5,main] (XID = 2734422), (SESSIONID = 0), (DATABASE
= pivotview), (DRDAID = null), Failed Statement is: select * from pivotview
> ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted to read
more data than was originally stored
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.StreamFileContainer.fetchNext(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.StreamFileContainerHandle.fetchNext(Unknown
Source)
> 	at org.apache.derby.impl.store.access.sort.MergeScan.mergeARow(Unknown Source)
> 	at org.apache.derby.impl.store.access.sort.MergeScan.init(Unknown Source)
> 	at org.apache.derby.impl.store.access.sort.MergeSort.openSortScan(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.openSortScan(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.loadSorter(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.openCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.ij.executeImmediate(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.utilMain.doCatch(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.utilMain.go(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.Main.go(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.Main.mainCore(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.Main14.main(Unknown Source)
> 	at org.apache.derby.tools.ij.main(Unknown Source)
> Caused by: java.io.EOFException
> 	at java.io.DataInputStream.readBoolean(DataInputStream.java:248)
> 	at org.apache.derby.impl.sql.execute.MaxMinAggregator.readExternal(Unknown Source)
> 	at org.apache.derby.iapi.services.io.FormatIdInputStream.readObject(Unknown Source)
> 	at org.apache.derby.iapi.types.UserType.readExternal(Unknown Source)
> 	... 22 more
> ============= begin nested exception, level (1) ===========
> java.io.EOFException
> 	at java.io.DataInputStream.readBoolean(DataInputStream.java:248)
> 	at org.apache.derby.impl.sql.execute.MaxMinAggregator.readExternal(Unknown Source)
> 	at org.apache.derby.iapi.services.io.FormatIdInputStream.readObject(Unknown Source)
> 	at org.apache.derby.iapi.types.UserType.readExternal(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.StreamFileContainer.fetchNext(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.StreamFileContainerHandle.fetchNext(Unknown
Source)
> 	at org.apache.derby.impl.store.access.sort.MergeScan.mergeARow(Unknown Source)
> 	at org.apache.derby.impl.store.access.sort.MergeScan.init(Unknown Source)
> 	at org.apache.derby.impl.store.access.sort.MergeSort.openSortScan(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.openSortScan(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.loadSorter(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.openCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
> 	at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(Unknown Source)
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.ij.executeImmediate(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.utilMain.doCatch(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.utilMain.go(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.Main.go(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.Main.mainCore(Unknown Source)
> 	at org.apache.derby.impl.tools.ij.Main14.main(Unknown Source)
> 	at org.apache.derby.tools.ij.main(Unknown Source)
> ============= end nested exception, level (1) ===========
> Cleanup action completed

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message