harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Egor Pasko <egor.pa...@gmail.com>
Subject Re: [drlvm][testing][jit] regular testing for JIT regressions on HUT
Date Thu, 28 Dec 2006 14:23:41 GMT
On the 0x24D day of Apache Harmony Geir Magnusson, Jr. wrote:
> On Dec 28, 2006, at 5:31 AM, Egor Pasko wrote:
> 
> >
> >> So, I think, these tests should go to different place, not to
> >> regression test suite. Right?
> >
> > Good point.
> >
> > Initially the intention was to call it JIT Unit Tests. But what can
> > stop us from tracking regressions with tests based on this idea?
> >
> > Let's look at the variants where to include the framework:
> > 1. smoke
> > 2. c-unit
> > 3. kernel
> > 4. regression
> > 5. harmony unit
> > 6. forgot something?
> > So, for the moment (4) seems most natural. Maybe, some tests to
> > include into (2) and some to (4). I do not insist, just want to know
> > your opinion for now.
> 
> TO be clear, 5 is "classlibrary tests", which I think should remain
> different.  

yep. I just tried to recall all test suites we have (and show that
it's pretty much for my small small brain organ)

> c-unit are also different, because it's a totally
> different testing framework (c-based). 

Well, actually, JIT IR-level tests are C-based too. Then, probably,
c-unit is the best place?

I think, it makes sense to combine *all* types of tests in a single
framefork with unified top-level running mechanisms as well as
reporting mechanisms. When we have them all together we can cut them
in pieces based on importance/time-to-run/catching-rate. Some pieces
to run as pre-commit some in CC, some even more rare on snapshots, etc.

> Now, that leaves smoke, kernel and regression.
> 
> Smoke and kernel should be able to be combined.  Smoke just seems to
> be tests where someone wanted to write their own testing framework.

I like that

> Regresssion -  I do understand where Tim is coming from in later
> messages in this thread, but I don't actually mind slapping tests
> that show reported bugs into a separate category called "regression",
> because they probably are more complicated than unit tests or even
> general functional tests, and a maybe poke at weird corner cases or
> effects of multiple interacting systems.  It would be interesting to
> see if our regression tests are finding things that unit tests should
> have...
> 
> geir
> 

-- 
Egor Pasko


Mime
View raw message