db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Nielsen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-1781) Process handles appear to be leaking in queries using an IN clause during concurrent DB access
Date Wed, 17 Oct 2007 07:36:50 GMT

    [ https://issues.apache.org/jira/browse/DERBY-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535437
] 

Thomas Nielsen commented on DERBY-1781:
---------------------------------------

Bryan, thanks for shedding some light on the DRDAConnThread lifespan. I was expecting them
to stay around, but I also was expecting them to be reused the second time I ran the repro
script. This does not seem to be the case, as I see new DRDAConnThreads being created, and
the old ones stay suspended. I guess there is a pool or a queue that these are placed into,
but are they only reused after N threads are created?

The majority of surviving generations are StoredRecordHeader and RecordId objects as previously
stated. Of the StoredRecordHeader objects 52% originate from store.raw.data.StoredPage.recordHeaderOnDemand()
and 47% from store.raw.data.StoredPage.storeRecordForInsert().
There's about 2200 live StoredRecordHeader objects consuming just above 100K of memory. This
is probably touching on the work Knut is currently doing in DERBY-3130.
The vast majority of RecordId objects are part of a StoredRecordHeader object, and originate
from calls to StoredRecordHeader.getHandle().

The number of surviving generations is normally a measure of probable memory leaks, but in
our case it does not seem to be the case at first sight. The StoredRecordHeader objects are
referenced from the StoredPage objects in the page cache. They are not freed until they are
evicted or cleared from the cache. They are intentionally kept from GC and will add to the
number of surviving generations. The pages are not cleared from the cache until the cache
is full and more pages need loading. The new pages will then have references to new StoredRecordHeader
object and the unused references will be GC'ed.

This investigation was done on the main trunk. I'll give 10.2.1.1 a spin as well to validate
that there are no significant changes here from 10.2.1.1 to trunk.

> Process handles appear to be leaking in queries using an IN clause during concurrent
DB access
> ----------------------------------------------------------------------------------------------
>
>                 Key: DERBY-1781
>                 URL: https://issues.apache.org/jira/browse/DERBY-1781
>             Project: Derby
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 10.1.3.1
>         Environment: Windows XP, Java 1.5.0_05
>            Reporter: Mark Hellkamp
>         Attachments: SqlStressTest.java
>
>
> We are currently using Derby embedded in our web application running on Windows. When
processing multiple concurrent requests we have noticed that the Java process handle count
continues to increase until the machine becomes unresponsive. I was able to isolate the problem
to Derby by running the database in network mode in another process. Further investigation
showed that the problem could be reproduced using a select statement that has an IN clause
with multiple entries on the primary key column. Spawning multiple threads running the same
query causes the handle count to increase considerably on the Derby process. The problem occurs
in version 10.1.3.1 and 10.2.1.1 (even worse) in both embedded and network mode. The attached
test program duplicates the problem. Start Derby in network mode (using startNetworkServer.bat)
and run the enclosed test program. The handle count on the Derby process will increase and
never go down short of restarting Derby. Using 10.2.1.1 the handle count for the Derby process
goes somewhere between 1400-1500 with just two threads in my environment. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message