db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-3024) Validation of shared plans hurts scalability
Date Fri, 24 Aug 2007 12:51:30 GMT

    [ https://issues.apache.org/jira/browse/DERBY-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12522495

Knut Anders Hatlen commented on DERBY-3024:

Thanks for investigating this, Manish!

I agree that the test is not representative of a real-world application, but that wasn't my
aim when I wrote it. I just wanted to see if there were any basic part of the SQL execution
layer that would be a bottleneck on a multi-CPU machine. VALUES 1 seemed to be a good choice
since it avoids accesses to the buffer manager, which is a known multi-CPU bottleneck. I think
of it more like looking at a small part of Derby through a magnifying glass or a microscope.

When I run the test, I only see three calls to GPS.upToDate(), one call to BA.checkStatementValidity(),
and none to GPS.finish() and GPS.getActivation(). You didn't by any chance use a Statement
instead of a PreparedStatement?

I'm not sure I quite understand how the interaction with upToDate() works. If upToDate() returns
true, we know (because of the synchronization) that at some point after we called upToDate()
and before it returned, the compiled plan was up to date. However, the synchronization doesn't
guarantee that the plan is up to date the moment after the method has returned, does it? How
do we know the plan is still valid then? Is it because of the uncertainty we keep calling
upToDate() multiple times during execution?

> Validation of shared plans hurts scalability
> --------------------------------------------
>                 Key: DERBY-3024
>                 URL: https://issues.apache.org/jira/browse/DERBY-3024
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, SQL
>    Affects Versions:
>         Environment: Sun Java SE 6, Solaris 10, Sun Fire V880 (8 CPUs)
>            Reporter: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: Values.java, values1.png
> To investigate whether there was anything in the SQL execution layer that prevented scaling
on a multi-CPU machine, I wrote a multi-threaded test which continuously executed "VALUES
1" using a PreparedStatement. I ran the test on a machine with 8 CPUs and expected the throughput
to be proportional to the number of concurrent clients up to 8 clients (the same as the number
of CPUs). However, the throughput only had a small increase from 1 to 2 clients, and adding
more clients did not increase the throughput. Looking at the test in a profiler, it seems
like the threads are spending a lot of time waiting to enter synchronization blocks in GenericPreparedStatement.upToDate()
and BaseActivation.checkStatementValidity() (both of which are synchronized on the a GenericPreparedStatement
> I then changed the test slightly, appending a comment with a unique thread id to the
"VALUES 1" statement. That means the threads still did the same work, but each thread got
its own plan (GenericPreparedStatement object) since the statement cache didn't regard the
SQL text strings as identical. When I made that change, the test scaled more or less perfectly
up to 8 concurrent threads.
> We should try to find a way to make the scalability the same regardless of whether or
not the threads share the same plan.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message