Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: pass (asf.osuosl.org: domain of mhn@cs.stanford.edu designates
 171.67.73.10 as permitted sender)
Date: Wed, 19 Oct 2005 00:54:27 -0700 (PDT)
From: Mayur Naik <mhn@cs.stanford.edu>
To: derby-dev@db.apache.org
cc: Mayur Naik <mhn@cs.stanford.edu>
Subject: synchronization issues in Derby client code
Message-ID: <Pine.LNX.4.62.0510190035180.3168@moa.Stanford.EDU>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed


Hi David,

Thanks for your feedback.  I ran my race detection tool on Derby's client 
code and here is a summary of the results:

http://suif.stanford.edu/~mhn/db-derby-10.1.1.0-src/classes/report.txt

The tool found 319 possible bugs (distinct instances of missing 
synchronization).  I will file all of them as a single JIRA bug item.

Note that I analyzed only the client code, not the engine code.  I hope to 
analyze the engine code as well at some point.

Your feedback is welcome!  In particular, I would be happy to hear from 
developers whether the races reported above are indeed bugs or harmless 
races.

I hope to release the tool sometime in the near future (it still lacks 
documentation, a good user interface, etc.).  Then, you should be able to 
run it on a regular basis.

-- Mayur

On 10/12/05, David W. Van Couvering wrote:
> 
> 
> Hello, Mayur.  This looks like *very* valuable information and I for one
> appreciate you using Derby as a test bed for your work.
> 
> It would be great to create a JIRA bug item for these race conditions
> (I'm not sure if one item per race is overkill or not), and yes,
> definitely we'd like to see more of this.
> 
> Is this tool something that could be run on a regular basis and provide
> reports?  It is likely that new race conditions could get introduced as
> new code is submitted...
> 
> Thanks,
> 
> David
> 
> Mayur Naik wrote:
> >
> > Hello,
> >
> > I'm not sure if this is the right mailing list to post this.  I'm a PhD
> > student in Computer Science at Stanford University and I'm evaluating a
> > static race detection tool I've developed on open source programs.  I
> > ran it to check the implementation of the java.sql.Connection interface
> > in Derby and found 76 races, see:
> >
> >
> http://suif.stanford.edu/~mhn/db-derby-10.1.1.0-src/connection_test/paos3/index.html
> >
> >
> > The races are due to 12 missing synchronization's in
> >
> > org/apache/derby/client/am/Connection.java
> >
> > In particular, the following methods need to be synchronized:
> >
> > boolean getAutoCommit()
> > boolean isClosed()
> > int getTransactionIsolation()
> > java.sql.SQLWarning getWarnings()
> > java.sql.DatabaseMetaData getMetaData()
> > boolean isReadOnly()
> > String getCatalog()
> > java.util.Map getTypeMap()
> > int getHoldability()
> > java.sql.PreparedStatement prepareStatement(String, int)
> > java.sql.PreparedStatement prepareStatement(String, int[])
> > java.sql.PreparedStatement prepareStatement(String, String[])
> >
> > Lack of synchronization in these methods can lead to unexpected
> > behavior, for instance, the race:
> >
> >
> http://suif.stanford.edu/~mhn/db-derby-10.1.1.0-src/connection_test/paos3/R432_trace.html
> >
> >
> > is between the accesses to field warnings_ on lines 1614 and 0865:
> >
> > 1610     public void accumulateWarning(SqlWarning e) {
> > 1611         if (warnings_ == null) {
> > 1612             warnings_ = e;
> > 1613         } else {
> > 1614             warnings_.setNextException(e);
> > 1615         }
> > 1616     }
> >
> > 0864     public void clearWarningsX() throws SqlException {
> > 0865         warnings_ = null;
> > 0866         accumulated440ForMessageProcFailure_ = false;
> > 0867         accumulated444ForMessageProcFailure_ = false;
> > 0868         accumulatedSetReadOnlyWarning_ = false;
> > 0869     }
> >
> > The latter method is called by clearWarnings(), which is synchronized,
> > but the former method is called by 2 prepareStatement methods along
> > paths which aren't sycnhronized.  In particular, you can follow the "up"
> > links in the above report and see that accumulateWarning(SqlWarning) is
> > called by downgradeResultSetType(int) which is called by
> > prepareStatementX(...) which is called by:
> >
> > java.sql.PreparedStatement prepareStatement(String, int)
> > java.sql.PreparedStatement prepareStatement(String, String[])
> >
> > Hence, if clearWarnings() and one of these prepareStatement methods are
> > executed concurrently in separate threads on the same Connection object,
> > there may be a null pointer exception.  I don't know if such scenarios
> > are possible, but the presence of synchronization in several methods in
> > Connection.java suggests so.
> >
> > I would be happy if you could confirm/refute these bugs.  Also, if you
> > are interested in hearing about additional possible synchronization
> > issues in Derby, I can check all of the Derby source code for races --
> > please let me know.
> >
> > Thanks!
> > -- Mayur
> >
>