harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [testing] Excluding tests until the problem is fixed (was: [drlvm] ... )
Date Fri, 24 Nov 2006 17:03:33 GMT
> I'll push back against TestNG here - because having a separate file
> means that you can answer questions like "How many tests have we had to
> exclude for platform X over time?" by simply looking at the diffs for
> that file over time rather than having to through each test over time

Well, this question should be clarified IMO. Many people think that we
are still waiting for TestNG. We can't keep distance from everything,
we need to choose one of the harnesses.

Regards,

23.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
>
>
> Alexei Zakharov wrote:
> > Tim Ellison wrote:
> >> >> >> Before you go off writing more code, just take a moment to
look at
> >> >> >> HARMONY-263 and tell us what you think of it.
> >
> > Geir Magnusson Jr. wrote:
> >> What I meant by my comment is that I'm interested in seeing excludes in
> >> external files, which can then be organized by platform, and have that
> >> information used at the high-level in ant.  So for that existing JIRA,
> >> reusing code for managing exclude lists is a good idea, but I'm still
> >> fairly resistant to the idea of tying ourselves closer to junit.
> >
> > I've looked through old HARMONY-263. In general it is close to the
> > approach that was suggested by me. But George (BTW there is George?)
> > proposed using rather complicated XML lists that IMHO can be not very
> > handy for using in real world.
> >
> > Junit part is rather small. Really, a few lines of code. The most part
> > of the code (located in tests.util.SomeTest) is devoted to parsing of
> > these complex XML lists by means of XPathAPI.  The most interesting
> > thing here is that this code is already in our repository and has been
> > lying there since March 2006.
> >
> > IMHO this all is not so much about "trying ourselves closer to junit".
> > This is more likely about do we like to keep the info about excludes
> > in a separate file or we want to have it together with source code as
> > TestNG suggests.
>
> I'll push back against TestNG here - because having a separate file
> means that you can answer questions like "How many tests have we had to
> exclude for platform X over time?" by simply looking at the diffs for
> that file over time rather than having to through each test over time.
>
> If things are embedded as annotations, we revert to the situation that
> we had with the DRLVM smoke tests, which was that excluding a test
> required modifying the test source, not the "control source".  I thought
> that was just awful - it meant that if I wanted to exclude on my machine
> for some reason, it meant I always had to be careful about not
> committing that modified file.
>
> With exclude lists, we can also have things like
>
>     make/exclude/exclude.local
>
> which would be svn-ignored, to allow localization w/o special handling
> on commits.


-- 
Alexei Zakharov,
Intel Enterprise Solutions Software Division
Mime
View raw message