commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: [codec][lang] Provide a test jar plus [daemon]
Date Mon, 05 Apr 2010 18:17:20 GMT
On 05/04/2010, Gary Gregory <GGregory@seagullsoftware.com> wrote:
> I do not think Clirr is enough as exemplified by the change of behavior in Base64 encoding
with CRLFs between version 1.3 and 1.4 (see CODEC-94.)

Agreed - Clirr is necessary, but not sufficient.

>  There is no substitute for running unit tests.

Agreed; however unit tests may not pick up changes found by Clirr.

Ideally the tests should exercise all public APIs, but that is not
always possible - and rarely achieved ;-)

>  As we change code in a component, unit tests are created or changed. It is possible
to create compatibility problems when code and unit tests change. You have to consciously
and carefully leave some tests as-is from a previous versions to ensure backward compatibility.
Sometimes of course, you are fixing bugs, so unit tests must change too.
>
>  If you run the previous version's test against the current code base, you are simulating
what users do when they drop in a new jar in their application. Since we do not have previous
version's tests next to the current tests, how can we test backward compatibility? If we produce
a test jar file for each release and keep it in SVN and deliver it, that is a snapshot of
old "clients" for a given version. The test phase of the build can then run all of the tests
in  each test jar, including the test jar for the current tests, against the component jar.
>

I agree, it would be useful to be able to run old tests against the new code.
However I'm not sure that the jars should be stored in SVN. Perhaps in
a local snapshot repo instead?

If there are a lot of bug fixes in the release, then a lot of old
tests may fail, and it could get tedious working out which failures
are OK and which are not. This is not to say that the tests should not
be run, just that they may involve a lot of manual checking.

>  Gary
>
>
>  > -----Original Message-----
>  > From: Matt Benson [mailto:gudnabrsam@gmail.com]
>  > Sent: Monday, April 05, 2010 10:00
>  > To: Commons Developers List
>
> > Subject: Re: [codec][lang] Provide a test jar plus [daemon]
>  >
>  > Is it not sufficient to simply run clirr reports before a release?
>  >
>  > -Matt
>  >
>  > On Apr 5, 2010, at 11:16 AM, Gary Gregory wrote:
>  >
>  > > Seeing the discussion about [daemon] and not releasing made me
>  > > think of another use for a test jar file.
>  > >
>  > > What I would like to know when evaluating an RC for releasing a
>  > > maintenance of a commons component (from x.y.n to x.y.n+1) is that
>  > > there is 100% binary compatibility.
>  > >
>  > > As part of the build I would run (at least) the 1.0.2 unit tests
>  > > against the 1.0.3 RC. If 100% pass all is well, if not, it is
>  > > either a bug or a known acceptable failure fixing a bug and should
>  > > be documented somehow, probably in a ticket.
>  > >
>  > > This would mean that a release 1.0.3 RC would include foo-
>  > > test-1.0.2.jar. And maybe also foo-test-1.0.0.jar and foo-
>  > > test-1.0.1.jar, hm...
>  > >
>  > > Thoughts?
>  > >
>  > > Gary Gregory
>  > > Senior Software Engineer
>  > > Seagull Software
>  > > email: ggregory@seagullsoftware.com
>  > > email: ggregory@apache.org
>  > > www.seagullsoftware.com
>  > >
>  > >
>  > > From: Gary Gregory
>  > > Sent: Tuesday, March 30, 2010 16:58
>  > > To: Commons Developers List
>  > > Subject: [codec][lang] Provide a test jar
>  > >
>  > > I am starting with codec and lang since it what I am most
>  > > interested in ATM...
>  > >
>  > > I would like to run commons.xxx unit tests as part of my build as a
>  > > sanity check when I try out a new combo of JVM, OS, jars, etc.
>  > >
>  > > Right now, I would have to compile the unit tests as part of my
>  > > build which is not great.
>  > >
>  > > So, what about providing a commons-codec-1.5-test.jar file like we
>  > > provide a sources and javadoc file?
>  > >
>  > > Same for any commons project really...
>  > >
>  > > Thoughts?
>  > >
>  > > Gary Gregory
>  > > Senior Software Engineer
>  > > Seagull Software
>  > > email: ggregory@seagullsoftware.com
>  > > email: ggregory@apache.org
>  > > www.seagullsoftware.com
>  > >
>  > >
>  >
>  >
>  > ---------------------------------------------------------------------
>  > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>  > For additional commands, e-mail: dev-help@commons.apache.org
>
>
>  ---------------------------------------------------------------------
>  To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>  For additional commands, e-mail: dev-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message