db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bergquist, Brett" <BBergqu...@canoga.com>
Subject RE: Question on unloading in an embedded environment
Date Wed, 17 Aug 2011 15:16:29 GMT
So this is running on Solaris 10 Java 1.6.0_22 with "-d64 -Xmx8129m -XX:+HeapDumpOnOutOfMemory"
  Unfortunately the heap dump is so large it take 32Gb for jhat to load it and then forever
for look at anything.  I am running the app again with a smaller heap to get the problem to
occur quicker and with a smaller heap dump to be able to get a allocation traceback.

-----Original Message-----
From: Bergquist, Brett [mailto:BBergquist@canoga.com] 
Sent: Wednesday, August 17, 2011 10:49 AM
To: derby-dev@db.apache.org
Subject: RE: Question on unloading in an embedded environment

This was run with 8gb of heap available.  The system is a Oracle M3000 with 32Gb of real ram.
 It did not run out of swap while running, so it hit the heap max.

Why would the sorter be called when doing a SYSCS_IMPORT_TABLE?  That is all this application
is doing.  It exports a table from one database with SYSCS_EXPORT_TABLE and imports into the
other database with SYSCS_IMPORT_TABLE.  No queries or other statements are being used.

-----Original Message-----
From: Bryan Pendleton [mailto:bpendleton.derby@gmail.com] 
Sent: Wednesday, August 17, 2011 9:40 AM
To: derby-dev@db.apache.org
Subject: Re: Question on unloading in an embedded environment

> Instance Counts for All Classes (excluding platform)
> 38006784 instances of class org.apache.derby.impl.store.access.sort.Node

One possibility is that the sorter is confused about how much memory
is available.

The sorter is very clever, and tries to figure out whether it can perform
the sort in-memory, or whether it has to switch to an external (disk-based)
sort, using temporary files to hold the partially-sorted data.

If the sorter is confused about how much memory is available (i.e., thinks
there is more memory available than there actually is), then it might
drive the system out of memory.

This would certainly be a bug, but not a leak exactly, rather a flaw
in the internal-vs-external sort analysis code.



P.S. I'm still wondering if your memory issues are actually external to
Derby; that is, if you've configured your JVM with memory sizes that exceed
the memory available in the underlying operating system. That would cause
Derby to attempt to allocate data structures that the JVM was willing to
allow, but which the underlying OS refused to provide.

View raw message