db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (DERBY-5206) Investigate reduced throughput in some of the nightly performance tests
Date Fri, 13 May 2011 07:24:48 GMT

     [ https://issues.apache.org/jira/browse/DERBY-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Knut Anders Hatlen closed DERBY-5206.
-------------------------------------

    Resolution: Invalid

I've had a very hard time reproducing this, but at last I remembered that the test itself
was changed in DERBY-5067 around the time that it started happening. So now, if I run the
test with Derby 10.8.1.2, I am able to see that the throughput is slightly higher if I take
derbyTesting.jar from 10.7.1.1 (before DERBY-5067) than it is with derbyTesting.jar from 10.8.1.2.

I only see this if I run the test with the -init parameter to create a new test table in the
same process that's running the actual test. If I run without -init, so that the test table
from a previous run is reused, I don't see any difference in the throughput.

The changes in DERBY-5067 only affect how the test table is created (the primary key constraint
is now added after the table has been populated, not when the table is created), which would
explain why we only see it when running with -init. But the exact reason why is a bit of a
mystery.

This issue sounds very similar to DERBY-4233. There we saw a drop in throughput in the table-scan
test (same test as here) only when -init was specified, and the change that triggered the
drop was a seemingly harmless change in a code path only executed when populating the test
table.

The theory in that issue was that the runtime compiler in the JVM would optimize some code
path based on statistics collected when populating the test table. The changes in how the
table was populated would change these statistics slightly, possibly making the optimizer
make different choices than before. Although these choices may be reasonable when populating
the table, they may not be that good when the test starts doing selects instead of inserts,
and it's the performance of the selects we're measuring in the test.

I think this is a theory that would explain what we're seeing in this issue too, and in any
case, since the performance drop happened after a change in the test itself, there shouldn't
be anything to fix in the product code. So I'll close the issue as invalid.

(After DERBY-4233 I was planning to change the nightly tests to create the test tables in
a separate process to avoid this noise. Still on my todo list...)

> Investigate reduced throughput in some of the nightly performance tests
> -----------------------------------------------------------------------
>
>                 Key: DERBY-5206
>                 URL: https://issues.apache.org/jira/browse/DERBY-5206
>             Project: Derby
>          Issue Type: Task
>          Components: SQL
>    Affects Versions: 10.8.1.2
>         Environment: Solaris 10 5/08 s10x_u5wos_10 X86
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build 1.6.0-b105)
> Java HotSpot(TM) Server VM (build 1.6.0-b105, mixed mode)
>            Reporter: Knut Anders Hatlen
>
> The last two months, the nightly performance regression tests (http://home.online.no/~olmsan/derby/perf/)
have shown results that are slightly lower than normal for the following test:
> - Single Record Select, Non-primary non-indexed column - http://home.online.no/~olmsan/derby/perf/select_sec_noindex_1y.html
> - TPC-B like load (aka bank_tx) - http://home.online.no/~olmsan/derby/perf/tpcb_1y.html
> For example, when averaging the results from 2011-03-01 to 2011-04-26 and comparing them
to the average for the two preceding months (2011-01-01 to 2011-03-01), I see these changes:
> Select, single-threaded: 17.05 tx/s -> 16.81 tx/s (down 1.4%)
> Select, 4 threads: 25.50 tx/s -> 24.70 tx/s (down 3.1%)
> Select, 16 threads: 25.17 tx/s -> 24.47 tx/s (down 2.8%)
> bank_tx, single-threaded: 1931 tx/s -> 1904 tx/s (down 1.4%)
> bank_tx, 4 threads: 2628 tx/s -> 2589 tx/s (down 1.5%)
> bank_tx, 16 threads: 3584 tx/s -> 3519 tx/s (down 1.8%)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message