db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel John Debrunner (JIRA)" <derby-...@db.apache.org>
Subject [jira] Commented: (DERBY-444) Handle OutOfMemoryError exceptions when creating a new embedded connection
Date Sat, 04 Mar 2006 00:32:41 GMT
    [ http://issues.apache.org/jira/browse/DERBY-444?page=comments#action_12368821 ] 

Daniel John Debrunner commented on DERBY-444:

More on the setting the low water mark. The low memory water mark is a value, proposed by
this fix, that  disables operations
if the JVM's free memory is lower than the mark, to allow some chance of recovery rather than
wasting cpu cycles performing
an operation that is most likely doomed to throw an OutOfMemoryError (OOME). The low water
mark would be set dynamically per operation type by checking the free memory at the first
OOME for that operation.

Given that I can successfully open around 19,000 connections in 64Mb, that makes a just opened
connection overhead on the order of  3.5kb.  I was suprised when the call  Runtime.getRuntime().freeMemory()
returned around 100k to 400k when an OutOfMemoryError  (OOME) is hit the first time opening
a connection. I was expecting a number closer to the apparent overhead of the connection.

Because in a multi-threaded environment another thread could release a whole lot of memory
between my thread's OOME and  it calling Runtime.getRuntime().freeMemory()  I was planning
to ignore the low water mark if was was "bigger" than some estimate, in this case say 10k
for a connection. This was to avoid a situation of setting a low water mark that was huge
(due to other released memory) and subsequently disabling all connection requests.

However the reality is the freeMemory at OOME time seems much bigger than any reasonable estimate,
and I'm not sure a estimate of 100k-400k for a connection seems reasonable when the overhead
is 3.5k. Seems like in some JVMs the low water
mark could typically be much lower.

My new approach is to only have the low water mark for 5 seconds, then it gets discarded,
any subsequent  OOME for the same operation would then set a new low water mark. This means
even a really bad low water mark would not break the system, but just deny service of that
operation for five seconds. I picked five seconds as a reasonable time for memory to be released,
say by future work in Derby.

I also tried running the  memory/ConnectionHandling test without the concept of a low water
mark, just throwing the
static SQLException if an OOME was thrown. This did not work, the test just hung somewhere
in the opening 500
extra connections once the first OOME was hit. Thus this backing off strategy with the low
water mark is some help,
it allows the JVM to spend some cpu cycles doing useful garbage collection, rather than trying
operations that are doomed to fail.

> Handle OutOfMemoryError exceptions when creating a new embedded connection
> --------------------------------------------------------------------------
>          Key: DERBY-444
>          URL: http://issues.apache.org/jira/browse/DERBY-444
>      Project: Derby
>         Type: Sub-task
>   Components: JDBC
>     Reporter: Daniel John Debrunner
>     Assignee: Daniel John Debrunner
>  Attachments: derby444_draft_v1.txt
> If an OutOfMemoryError is thrown while creating objects for a new embedded connection,
reject the connection request with a SQLException and do not shutdown the system.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message