db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DERBY-5406) Intermittent failures in CompressTableTest and TruncateTableTest
Date Mon, 24 Oct 2011 08:42:32 GMT

     [ https://issues.apache.org/jira/browse/DERBY-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Knut Anders Hatlen updated DERBY-5406:

    Attachment: d5406-3a.diff

I mentioned that I still saw similar stack traces when the compilation was
invoked from GenericActivationHolder.execute() instead of
GenericPreparedStatement.executeStmt(), and suggested that
GenericActivationHolder needed retry logic similar to the one in

The attached patch (d5406-3a.diff) takes a somewhat different approach. Instead
of adding the extra logic to GenericActivationHolder, it makes
GenericActivationHolder.execute() stop re-preparing the statement if it detects
that it's using an outdated generated class. Instead, it just asks the prepared
statement to give it the most recent version of the generated class.

In most cases, it will receive an up-to-date version of the class, and it can
continue without recompiling (the existing code would short-circuit the
rePrepare() call in that case, so no changes in this scenario).

If an invalidation happened after the last recompilation of the statement, the
fresh version of the generated class will also be outdated. With the existing
code, a recompilation would be requested immediately. With the patch, however,
we just go ahead executing using the outdated class. The execution code already
has checks for invalid plans, so it will be detected by the normal execution
mechanisms. This has the advantage that the invalid plans will be reported in a
way that GenericPreparedStatement.executeStmt() is able to detect, and the
recompilation will be done by GenericPreparedStatement.executeStmt(). Since we
already have the required retry logic in place there, re-invalidation of the
statement during the recompilation will be detected and handled properly.

(This is, by the way, the exact same thing as the existing code would do if the
invalidation had happened right after we had fetched the fresh class. So this
change could be seen as handling the two cases - invalidation right before
retrieving the class and invalidation right after retrieving the class -

Another edge case is that the returned generated class could be null. This
happens if another thread was recompiling the statement when we retrieved the
class. In that case, the patch makes GenericActivationHolder.execute() throw an
exception with message id LANG_STATEMENT_NEEDS_RECOMPILE. This is a special
kind of exception that GenericPreparedStatement.executeStmt() detects and takes
as a signal to recompile the statement. Again, the recompilation will happen
using the code that's already prepared for the need to retry in case of
re-invalidations, so we should be covered if the conglomerate disappears during
that compilation too.

This also has the benefit that we can remove the workaround for DERBY-3260,
where we added a synchronization block around the calls to rePrepare() and
getActivationClass() to prevent that a concurrent recompilation made
getActivationClass() return null.

All the regression tests ran cleanly with the patch.

I also ran my standard test case, four parallel processes of the D4275 class,
for two hours without seeing any failures.
> Intermittent failures in CompressTableTest and TruncateTableTest
> ----------------------------------------------------------------
>                 Key: DERBY-5406
>                 URL: https://issues.apache.org/jira/browse/DERBY-5406
>             Project: Derby
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions:,
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>         Attachments: d5406-1a-detect-invalidation-during-compilation.diff, d5406-1b.diff,
d5406-2a-invalidate-self.diff, d5406-3a.diff
> The test cases CompressTableTest.testConcurrentInvalidation() and TruncateTableTest.testConcurrentInvalidation()
fail intermittently with errors such as:
> ERROR XSAI2: The conglomerate (2,720) requested does not exist.
> The problem has been analyzed in the comments on DERBY-4275, and a patch attached to
that issue (invalidation-during-compilation.diff) fixes the underlying race condition. However,
that patch only works correctly together with the fix for DERBY-5161, which was backed out
because it caused the regression DERBY-5280.
> We will therefore need to find a way to fix DERBY-5161 without reintroducing DERBY-5280
in order to resolve this issue.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message