db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kristian Waagan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-4771) Continue investigation of automatic creation/update of index statistics
Date Thu, 11 Nov 2010 13:28:13 GMT

     [ https://issues.apache.org/jira/browse/DERBY-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Kristian Waagan updated DERBY-4771:

    Attachment: derby-4771-2c-prototype_lcc_code_dump.diff

Attaching patch 2c.

The two main changes:
 a) Removed code in AlterTableConstantAction for updating statistics, and made it use the
 b) Removed status tracking code in TableDescriptor/statistics object.

The removal of code for (b) may cause some extra calls to the daemon, and the scheduling requests
will be rejected. I'm not sure how critical this is yet, which is why I decided to remove
the code. Maybe a more light-weight mechanism can be re-added later.

One major concern I have right now, is how to use the dependency system.
When should the invalidation request be performed?
Does it have to be before the statistics have been generated, or can it be done afterwards?
If the table is large, it may take a long time from the invalidation request is performed
until the index(es) has been scanned and the new statistics added to the system tables.

I also see a problem when running CheckToursDBTest, where we end up in a kind of  deadlock
situation. There are two threads, one daemon thread (D) updating statistics, and one user
thread (U) updating a table with some triggers. Depending on timing, the following may happen:
  D: takes S-locks on some rows in SYSDEPENDS for invalidation
  U: compiles trigger/statement, enters DependencyManager.clearDependencies() and grabs the
monitor 'this'.
  D: needs to enter DM.clearDependencies(), but U already has the monitor 'this' (BLOCKED)
  U: needs to get X-locks on the rows already locked by D in SYSDEPENDS (TIMES OUT)
  D: after U is killed due to a lock timeout, D can continue and successfully complete the
I have not yet figured out how/if this issue can be resolved.
Also, IndexStatisticsDaemon has to be moved to a different package, or at least a new interface
has to be introduced if the DataDictionary is used to access the daemon. Any opinions on where
to place and how to access the daemon?

I'm asking for review, particularly of the code in IndexStatisticsDaemon. The changes in impl/sql
would also benefit from review from someone who knows more about the dynamics here.
When I run suites.All I see from zero to three failures on my machine. I've also seen some
failures in derbyall earlier, but I haven't run it with my latest changes (will do).

Patch ready for review.

> Continue investigation of automatic creation/update of index statistics
> -----------------------------------------------------------------------
>                 Key: DERBY-4771
>                 URL: https://issues.apache.org/jira/browse/DERBY-4771
>             Project: Derby
>          Issue Type: Task
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>         Attachments: derby-4771-1a-prototype_code_dump.diff, derby-4771-1a-prototype_code_dump.stat,
derby-4771-1b-prototype_code_dump.diff, derby-4771-2a-prototype_lcc_code_dump.diff, derby-4771-2b-prototype_lcc_code_dump.diff,
derby-4771-2c-prototype_lcc_code_dump.diff, derby.log, error-stacktrace.out, rjall.out, rjall.out,
rjall.rar, rjone.out
> Work was started to improve Derby's handling of index statistics. This issue tracks further
discussion and work for this task.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message