db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John H. Embretsen (JIRA)" <derby-...@db.apache.org>
Subject [jira] Updated: (DERBY-1713) Memory do not return to the system after Shuting down derby, following an out of memory event
Date Thu, 31 Aug 2006 14:46:23 GMT
     [ http://issues.apache.org/jira/browse/DERBY-1713?page=all ]

John H. Embretsen updated DERBY-1713:

    Attachment: Derby1713repro.java

I have not been able to try Ibrahim's Test1.java, since I am not able to decompress the database
needed (my version of bzip2 tells me it (test.zip) is not a valid bzip2 archive - and I'm
not using Windows (at the moment)).

Instead, I have created a fully reproducible test case which does not require having a database
in advance. See attachment 'Derby1713repro.java'. To run, put derby.jar in your classpath
and run:

$ java Derby1713repro

If you already have a DB to test with, you can run with the "haveDB" argument:

$ java Derby1713repro haveDB

The test case inserts 40k rows into a table (same DDL as Ibrahim described in previous comments),
which results in a 30 MB database (unless "haveDB" is specified, in which case the test case
only performs a query against an already existing database).

I think one important thing to do is to reduce as many variables as possible when trying to
reproduce or debug a problem. With this repro, I do get an OutOfMemoryError (OOME) if the
max heap size is 32 MB (-Xmx32M). This happens without using an inner class (as is done in
Test1.java), and it even happens without executing a 'SELECT count(*)' statement. I have also
tried commenting out the setFetchSize(60000) call, and the OOME still happens (Sun's jvm version

The problem seems to be the huge SELECT, FROM, WHERE statement. At start-up (when using the
"haveDB" option), jstat reports a tenured space usage of 512 bytes. When executing the query,
tenured space usage increases to as much as 30272 bytes, which is almost the whole heap. In
addition, other parts of the heap need some space, hence we get an OutOfMemoryError. Feel
free to use this repro as basis for creating a minimalistic test showing the problem.

I think Mike is right when he says that 32 MB max heap is not suitable ("safe") for this kind
of database and this kind of usage. I therefore recommend lowering the Priority and Urgency
of this Jira issue somewhat.

I also recommend changing the summary of this issue once the real problem has been positively
identified. The fact that the JVM is not able to free memory following an OutOfMemoryEvent
is as far as I can tell not a bug in Derby.

> Memory do not return to the system after Shuting down derby, following an out
of memory event
> ------------------------------------------------------------------------------------------------------
>                 Key: DERBY-1713
>                 URL: http://issues.apache.org/jira/browse/DERBY-1713
>             Project: Derby
>          Issue Type: Bug
>          Components: Performance
>    Affects Versions:
>         Environment: Windows XP SP2
> JRE 1.6 beta2
>            Reporter: Ibrahim
>            Priority: Critical
>         Attachments: Derby1713repro.java, test.zip, Test1.java
> I face a problem when querying large tables. I run the below SQL and it stuck in this
query and throws java heap exception OutOfMemory:
> SELECT count(*) FROM <table> WHERE .....
> N.B. I'm using a database of more than 90,000 records (40 MB). I set the maxHeap to 32
MB (all other settings have the default value, pageCache ... etc ). 
> Then, I shutdown the database but the memory is not returned to the system (and remain
32 MB [max threshold]). I tried to increase the maxHeap to 128 MB in which it works and releases
the memory, so I think the problem is when it reaches the maxHeap then it seems to not respond
to anything such as closing the connection or shutting down the database. How can I get rid
of this? (because i cannot increase the maxHeap as the database increases, I want to throw
an exception and release the memory)
> I'm using this to shutdown the DB:
> try{DriverManager.getConnection("jdbc:derby:;shutdown=true");}
> catch(SQLException ex){System.err.println("SQLException: " + ex.getMessage());}
> I'm using a memory Profiler for monitoring the memory usage.
> Thanks in advanced.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message