Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Derby Development" <derby-dev@db.apache.org>
Received-SPF: neutral (hermes.apache.org: local policy)
Date: Thu, 26 May 2005 21:36:42 -0700
From: David Van Couvering <david@vancouvering.com>
Subject: Re: Running derbyall before submitting patches
In-reply-to: <42961EAC.8000803@sbcglobal.net>
To: Derby Development <derby-dev@db.apache.org>
Message-id: <4296A3DA.4050705@vancouvering.com>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii; format=flowed
Content-transfer-encoding: 7BIT
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6)
 Gecko/20040113
References: <20050526112723.GE22123@atum01.norway.sun.com>
 <42961261.3040607@Sourcery.Org> <42961EAC.8000803@sbcglobal.net>

Hm, it's tricky to just count on the developer submitting the patch to 
run derbyall.  A common mistake, for example, is that the developer 
hasn't done 'svn add' for a new file, so their tests work, but the patch 
submitted doesn't have all the necessary files.  Also, the developer may 
have old classes sitting around or who knows what funny stuff in their 
environment that makes it work for them.  Also also, the developer 
usually tests on a specific VM and OS, and it would be great if at least 
one other VM or OS were tried before committing.

What we *really* need is a good full nightly regression across all 
platforms and VMS, but until we have that, I think we need to have at 
least *one* person run derbyall besides the submitter of the patch. 
This does not necessarily have to be a committer.

I'm not sure how this could be managed; perhaps each developer could 
"buddy up" with someone, and each of them agree to test out each other's 
patches...

The other thing I have noticed is that sometimes a developer forgets to 
run 'svn update' and 'ant clobber' and 'ant all' before running tests.

We should probably write up a "checkin checklist", vote on it, and add 
it to the web site...

David

P.S. I did run derbyall on the two recent patches I submitted.  I'll add 
that to my comments when I submit patches from now on.

Mike Matrigali wrote:

> me too.  I run derbyALL for any thing I submit myself, but as
> satheesh has said lately I have not been able to run derbyAll for
> all the patches I have been committing for others.  I do a full
> rebuild, review the change and
> run tests that look specific to the patch, and count on the submitter
> to have run all tests.  Also please check the results posted to
> the list the day after your patch goes in, to see if you may have
> introduced a problem in a different environment.
> 
> Remember that there are now a lot of developers out there depending
> on clean test runs, so they can figure out what affect their changes
> are having.
> 
> Satheesh Bandaram wrote:
> 
> 
>>+10, if there is anything like that. :-)
>>
>>I used to run 'derbyAll' for each patch I tried to commit, but not able
>>to do that anymore with so many patches being submitted these days. It
>>should be everyone's responsibility to run all tests before submitting a
>>patch. I will start asking for list of tests run before committing
>>patches, if not provided already.
>>
>>Also, new people tend to miss updating multiple versions of master
>>files, if the test has different masters. (say for DerbyNet or
>>DerbyNetClient frameworks) It is also possible to have different master
>>files based on JVM being used or version.
>>
>>Satheesh
>>
>>Bernt M. Johnsen wrote:
>>
>>
>>
>>>Hi all,
>>>
>>>The last week or two, derbyall has failed a lot the time. This makes
>>>it hard to verify that I haven't goofed up things when doing changes
>>>in the code (I run derbyall daily as a quality measure). To be able to
>>>work efficiently, many of us are dependent on a reliable test suite.
>>>
>>>So, I urge everyone who submits a patch to the Derby to run derbyall
>>>on at least one platform/vm and not only the "relevant" tests for the
>>>patch, even if you "know" that it should not be necessary. I think
>>>this would save us all a lot of work (even more time saved for the
>>>committers, I guess :-).
>>>
>>>I know this won't guarantee that derbyall runs correctly all the time,
>>>but the frequency of problems should at least go down.
>>>
>>>Thanks
>>>
>>>
>>
>>
>>