db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Nielsen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-1781) Process handles appear to be leaking in queries using an IN clause during concurrent DB access
Date Wed, 17 Oct 2007 11:48:50 GMT

    [ https://issues.apache.org/jira/browse/DERBY-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535514
] 

Thomas Nielsen commented on DERBY-1781:
---------------------------------------

The closest thing to 10.2.1.1 I had available was 10.2.1.6.

I reran the repro a single time on 10.2.1.6 and something has changed a lot from 10.2.1.6
and up the main trunk (10.4.0.0 alpha).

Max heapsize after *multiple* repro runs was close to 12Mb with main trunk (10.4.0.0 alpha),
and tipped 17Mb on the *first* run with 10.2.1.6.
Max number of surviving generations was on the order of 600, dropping to 430 on the main trunk
(10.4.0.0 alpha), but was increasing rapidly until 2000, then dipped to 600, and then steadily
increased until 849 on 10.2.1.6.
The main trunk feels a lot faster than 10.2.1.6 to execute the N transactions/iterations it
is supposed to do, but I haven't timed either.

The memory usage and generation counts on the main trunk are probably as expected (see above).

There are however two places in BasePage and StoredPage where the GC could be let a helping
hand by possibly rewriting constructs like 
[BasePage.java @ 1379]:
   for (int slot = FIRST_SLOT_NUMBER ; slot < maxSlot; slot++) {
      if (recordHeaderOnDemand(slot).isDeleted())
         delCount++;
   }

to something like:
   StoredRecordHeader rh = null;
   for (int slot = FIRST_SLOT_NUMBER ; slot < maxSlot; slot++) {
      rh = recordHeaderOnDemand(slot);
      if (rh.isDeleted())
         delCount++;
      rh = null;
   }

to explicitly tell the GC that the StoredRecordHeader returned from recordHeaderOnDemand()
can be gc'ed.

The latest official release, 10.3.1.4, is even more well behaved wrt to surving generations
than the main trunk it seems. First repro run gave "only" 125 surviving generations. Only
exception is there's an int[] originating in DataValueFactoryImpl.setLocale() that's eating
some memory which seem to have been fixed/changed as part of the collation work(?) on the
main trunk.
And there's only 59 or so surviving generations of the StoredRecordHeader object in 10.3.1.4,
opposed to ~400 on the main trunk.

To recap findings this far:
- the main trunk does not show the behavior reported.
- 10.2.1.6 does show an increased resource usage over the main trunk, but not to the point
of becoming unresponsive.
- an increasing "surviving generation count" seems plausible given the nature of the implemented
cache.
- Suggest user move to latest release, if possible, and reopen issue if it is still a problem.

> Process handles appear to be leaking in queries using an IN clause during concurrent
DB access
> ----------------------------------------------------------------------------------------------
>
>                 Key: DERBY-1781
>                 URL: https://issues.apache.org/jira/browse/DERBY-1781
>             Project: Derby
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 10.1.3.1
>         Environment: Windows XP, Java 1.5.0_05
>            Reporter: Mark Hellkamp
>         Attachments: SqlStressTest.java
>
>
> We are currently using Derby embedded in our web application running on Windows. When
processing multiple concurrent requests we have noticed that the Java process handle count
continues to increase until the machine becomes unresponsive. I was able to isolate the problem
to Derby by running the database in network mode in another process. Further investigation
showed that the problem could be reproduced using a select statement that has an IN clause
with multiple entries on the primary key column. Spawning multiple threads running the same
query causes the handle count to increase considerably on the Derby process. The problem occurs
in version 10.1.3.1 and 10.2.1.1 (even worse) in both embedded and network mode. The attached
test program duplicates the problem. Start Derby in network mode (using startNetworkServer.bat)
and run the enclosed test program. The handle count on the Derby process will increase and
never go down short of restarting Derby. Using 10.2.1.1 the handle count for the Derby process
goes somewhere between 1400-1500 with just two threads in my environment. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message