harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Egor Pasko <egor.pa...@gmail.com>
Subject Re: [drlvm][testing][jit] regular testing for JIT regressions on HUT
Date Thu, 28 Dec 2006 10:35:56 GMT
On the 0x24D day of Apache Harmony Alexey Varlamov wrote:
> 2006/12/28, Alexander Kleymenov <kleymenov@gmail.com>:
> > > BTW, Mikhail, are we going to integrate IR-level JIT regression tests
> > > ([1]) into the *DRLVM regression testing infrastructure* in the
> > > nearest future?
> >
> > Stop, guys, let's think. Regression test is a test checking for
> > particular problem reported and fixed before. So regression test and
> > JIRA report have one-to-one correspondence. What I see at HARMONY-1531
> > is a set of tests for Jitrino.OPT marked as "improvement", not as a bug.
> > So, I think, these tests should go to different place, not to regression
> > test suite. Right?
> 
> Alexander,
> I'm not expert in testing taxonomy, could you please share the full
> picture - which tests go where and when/how intended to be run, etc?
> 
> > > Are there any pending tasks to reach that point? IMHO, we should
> > > integrate/commit it and let go for constant improvement. That's quite
> > > enough for the start. (Not the highest priority, but gonnabe pretty
> > > useful). Please, tell me if I am missing something here.
> >
> > I'm agree with you. Tests first! Any tests increase the quality
> > of the product (if they do not do some intentional harm of course).
> 
> I got an impression that great diversity of test suites may come into
> conflict with this manifested goal - more frameworks to maintain, more
> steps to launch/track various tests, scattered reports, more complex
> desicion tree to add new tests, etc.
> Leaving aside official certification tests, I personally do not really
> care where from a test case is taken - from spec perseption of code
> implementer or from real-life user problem, the essence point is that
> it properly exercises particular bit of functionality. And further
> grading IMO should be practical tradeoff between
> importance/coverage/time to test, dividing pre-commit/pre-contribute
> and nightly or weekly CC runs.
> Ideally there should be just one entry point which takes in a
> config/testbase to be tested and a single consolidated test status and
> report.

"one entry point" for all tests sounds good

-- 
Egor Pasko


Mime
View raw message