db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francois Orsini <francois.ors...@gmail.com>
Subject Re: Cleaning up test failures
Date Fri, 20 Jan 2006 17:30:59 GMT
+1

On 1/20/06, David W. Van Couvering <David.Vancouvering@sun.com> wrote:
>
> Great email, John.  May I summarize a potential proposal.  Do people
> feel we need a vote on this?
>
> - Create a new alias, derby-test-results, that send *all* test results
>
> - Send test *failures* to derby-dev
>
> - Have a general guideline that test failures should be logged under the
> Test component with a high priority (I am using Critical for
> cross-platform issues and Major for single-platform issues).
>
> - Ultimately committers are responsible for maintaining the quality of
> the codeline and they may take further action where necessary (such as
> vetoing any checkins except those related to test fixes).
>
> Sound good?
>
> David
>
> John Embretsen wrote:
> > David W. Van Couvering wrote:
> >
> >> I would like to have an email list set up that sends out test results,
> >> for those of us who would like a "push" model to see what's going on.
> >> Personally I would want to see test failures sent to derby-dev.  If
> >> you don't like seeing them, you can filter them out, but I think it's
> >> good for the developers to have awareness of what's going on.
> >>
> >> I think that combined with logging JIRA issues would be good.  We
> >> could use Kathey's idea to get a report on open test bugs by filtering
> >> on the right component.
> >
> >
> > If it is agreed upon that we need to change the way developers get
> > alerted about test failures, I basically support what what is quoted
> > above. I've seen many good suggestions the past 24 hours for how to
> > increase awareness and potentially trigger action when a test in
> > derbyall fails. However....
> >
> > With regards to the _awareness_ part of the challenge, what we need
> > (IMHO) more than yet another wiki-page or other kind of web page that
> > people have to manually update and/or navigate to, is some kind of
> > "push" mechanism such as automatically sending e-mails to some mailing
> > list.
> >
> > Adding JIRA-entries sounds good to me, since e-mails are sent to
> > derby-dev when issues are created/updated (although this strategy seemed
> > a bit chaotic to me yesterday), thus increasing awareness. But, it
> > requires a certain amount of *manual* work. Someone has to have the itch
> > to check the actual test results (whether they are provided by Sun or
> > IBM or someone else), pick a test failure (if any exist), check if it
> > has been reported in JIRA already, and, if not, create a new JIRA issue,
> > etc. Because of this, I do not think this strategy is sufficient as a
> > long-term solution (but it may be a part of it).
> >
> > I suggest that we focus on deciding:
> >
> >  a) Do we want to get automated e-mails reporting test results?
> >
> >  b) If so, how often should it be reported (after every test run, after
> >   every test run with failures, after every test run with a certain
> > number of failures, once a day, once a day if failures occured, ...)?
> > (I guess the number of sources producing public test reports regularly
> > may be a factor here in the long run, but today there is only one such
> > source)
> >
> >  c) Where should such e-mails be sent (derby-dev, derby-test-results,
> > derby-commits, ...)?
> >
> > ...and that we try to keep this separate from the discussions of adding
> > new web-pages with test results, wiki-pages with test results, new
> > JIRA-types, etc.
> >
> >
>
>
>

Mime
View raw message