db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-3024) Validation of shared plans hurts scalability
Date Mon, 18 Jan 2010 14:01:55 GMT

     [ https://issues.apache.org/jira/browse/DERBY-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Knut Anders Hatlen updated DERBY-3024:

    Attachment: patch-2a.png

EmbedStatement.executeStatement() calls rePrepare() on the execution plan before trying to
execute it. Later, in the same method, the plan is executed by calling GenericPreparedStatement.execute().
One of the first things GPS.execute() does, is to call rePrepare().

Although calling rePrepare() on an already prepared and valid plan is basically a no-op, it
does involve a check of statement validity and therefore contributes to the problem reported
in this issue. It seems to me that the first call to rePrepare() is unnecessary, since the
statement will be re-prepared if needed right before execution anyways.

The attached patch (2a) removes the call to rePrepare() in EmbedStatement.executeStatement().
It also moves the retrieval of compile-time warnings until execute() has been called, since
re-preparation will now happen in execute().

All the regression tests ran cleanly with the patch. I also reran the Values test on a machine
similar to the one used for testing patch 1a. The attached graph patch-2a.png shows the performance
compared with trunk (which includes the 1a patch) and (which contains none of the
patches). There's still a long way to go before the scalability is similar to the one we get
when the threads don't share the same execution plan, but the patch does seem to improve the
situation somewhat.

> Validation of shared plans hurts scalability
> --------------------------------------------
>                 Key: DERBY-3024
>                 URL: https://issues.apache.org/jira/browse/DERBY-3024
>             Project: Derby
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions:
>         Environment: Sun Java SE 6, Solaris 10, Sun Fire V880 (8 CPUs)
>            Reporter: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: patch-1a.diff, patch-1a.png, patch-2a.diff, patch-2a.png, Values.java,
> To investigate whether there was anything in the SQL execution layer that prevented scaling
on a multi-CPU machine, I wrote a multi-threaded test which continuously executed "VALUES
1" using a PreparedStatement. I ran the test on a machine with 8 CPUs and expected the throughput
to be proportional to the number of concurrent clients up to 8 clients (the same as the number
of CPUs). However, the throughput only had a small increase from 1 to 2 clients, and adding
more clients did not increase the throughput. Looking at the test in a profiler, it seems
like the threads are spending a lot of time waiting to enter synchronization blocks in GenericPreparedStatement.upToDate()
and BaseActivation.checkStatementValidity() (both of which are synchronized on the a GenericPreparedStatement
> I then changed the test slightly, appending a comment with a unique thread id to the
"VALUES 1" statement. That means the threads still did the same work, but each thread got
its own plan (GenericPreparedStatement object) since the statement cache didn't regard the
SQL text strings as identical. When I made that change, the test scaled more or less perfectly
up to 8 concurrent threads.
> We should try to find a way to make the scalability the same regardless of whether or
not the threads share the same plan.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message