db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DERBY-5472) Speed up MemoryLeakFixesTest.testRepeatedDatabaseCreationWithAutoStats()
Date Wed, 19 Oct 2011 16:53:10 GMT

    [ https://issues.apache.org/jira/browse/DERBY-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130759#comment-13130759

Knut Anders Hatlen commented on DERBY-5472:

Hi Bryan,

I didn't benchmark them individually, but here's approximately what I saw as I added them:

With no changes: 85 sec for the embedded case

2+3+4 reduced it to 10 sec

1+2+3+4 reduced it to 5 sec

1+2+3+4+5 reduced it to 4.5 sec (don't know if the difference between 5 and 4.5 is significant
or just variation...)

So it's difficult to say if the improvements are mostly because of the reduced number of iterations,
or if the reduced compilation contributed. Disabling auto-commit didn't seem to do much, but
I ran the test on a file system where synchronous writes had been disabled, so the cost of
the extra commits cannot really be seen from these experiments.

I also had a hidden agenda, as I hope that removing the excessive recompilation will allow
us to remove the work-around for DERBY-5412 in the test case, but I haven't verified that

But now you made me curious, so I guess I'll have to run the experiments to see how costly
the compilation was. :)

If it turns out compiling insert statements actually contributes to the time it takes to run
the test, OrderByAndSortAvoidanceTest would be the next test to look at. That test has thousands
of hard-coded insert statements in the decorator, each of them needs to be compiled separately.
> Speed up MemoryLeakFixesTest.testRepeatedDatabaseCreationWithAutoStats()
> ------------------------------------------------------------------------
>                 Key: DERBY-5472
>                 URL: https://issues.apache.org/jira/browse/DERBY-5472
>             Project: Derby
>          Issue Type: Improvement
>          Components: Test
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>         Attachments: d5472.diff
> MemoryLeakFixesTest.testRepeatedDatabaseCreationWithAutoStats() takes fairly long time.
When I run it with -Xmx16M (per instructions in the comments) in my environment, that test
case alone takes 80-90 seconds. And it runs twice (embedded and client) so it takes nearly
3 minutes in total.
> There are ways to speed it up and still have it expose DERBY-5336, for which it was originally
> 1) The body of the test is executed 50 times. When the fix for DERBY-5336 is backed out,
the test case typically fails in the 8th or 9th iteration, so 20 iterations should be enough.
> 2) In each iteration, a table with 500 rows is created. Since the goal is to get the
istat daemon to run, we only need to insert enough rows to exceed derby.storage.indexStats.debug.createThreshold,
which is 100 by default. Reducing the size to for example 200 rows would be OK.
> 3) When populating the table, a select statement is compiled and executed in between
each insert statement. To get the istat daemon running, it's sufficient to execute a single
select statement once the table is populated.
> 4) Every insert statement is compiled separately. Would be better to compile once and
execute multiple times.
> 5) Populating the table could happen with auto-commit off.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message