Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: domain of gcjhd-harmony-dev@m.gmane.org
 designates 80.91.229.2 as permitted sender)
To: dev@harmony.apache.org
From: Egor Pasko <egor.pasko@gmail.com>
Subject: Re: [drlvm][testing][jit] regular testing for JIT regressions on HUT
Date: 28 Dec 2006 16:31:44 +0600
Lines: 79
Message-ID: <u4prgux9r.fsf@gmail.com>
References: <uy7otbnre.fsf@gmail.com>
 <bc79dd600612270624t131c77cfp28cd455b2838e7ae@mail.gmail.com>
 <umz58vdf1.fsf@gmail.com>
 <e09a11790612272205p6e52f008h2d9f6ecbbf5f27df@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3
Sender: news <news@sea.gmane.org>

On the 0x24D day of Apache Harmony Alexander Kleymenov wrote:
> Hello,
> 
> > OK))
> >
> > BTW, Mikhail, are we going to integrate IR-level JIT regression tests
> > ([1]) into the *DRLVM regression testing infrastructure* in the
> > nearest future?
> 
> Stop, guys, let's think. Regression test is a test checking for
> particular problem reported and fixed before. So regression test and
> JIRA report have one-to-one correspondence. 

Agreed.

> What I see at HARMONY-1531 is a set of tests for Jitrino.OPT marked
> as "improvement", not as a bug. 

yes, it is an improvement *that allows* to run special JIT IR-level
tests to run. Basic idea came from the fact that some JIT bugs can NOT
be reproduced by a simple Java test, we have Jasmin to write more
JIT-oriented tests, but many issues can NOT be covered by Jasmin
coding too. For example, quality of Register Allocator and such.

We would like to have more fine-tuned tests that create certain IR
(=Intermediate Representation) sources and feed JIT with them via a
special interface, then check, how JIT optimized it.

That's a technical detail. Now back to politics:

> So, I think, these tests should go to different place, not to
> regression test suite. Right?

Good point.

Initially the intention was to call it JIT Unit Tests. But what can
stop us from tracking regressions with tests based on this idea?

Let's look at the variants where to include the framework:
1. smoke
2. c-unit
3. kernel
4. regression
5. harmony unit
6. forgot something?
So, for the moment (4) seems most natural. Maybe, some tests to
include into (2) and some to (4). I do not insist, just want to know
your opinion for now.

ps: gosh, what a lots of test suites! how canitbe?

> > Are there any pending tasks to reach that point? IMHO, we should
> > integrate/commit it and let go for constant improvement. That's quite
> > enough for the start. (Not the highest priority, but gonnabe pretty
> > useful). Please, tell me if I am missing something here.
> 
> I'm agree with you. Tests first! Any tests increase the quality
> of the product (if they do not do some intentional harm of course).
> 
> > Alexander Kleymenov, are you interested? BTW, do you have a
> > prioritized list of TODO items for DRLVM RTI?
> 
> No, I don't have such. I see some further works on reg test suite (such as
> old reg tests converting/integration), but I did not make any plans and
> prioritizations. I think we should have public list of such TODOs so
> everyone interesting could help us with it. Your thoughts?

Yeah, knowing the prioritized list is very important. Let's create it
and publish somewhere, say, on wiki to track things in a better way.

What I see immediately:
* exclude lists
* integration of HARMONY-1531 // we are discussing it
* integration to other test suites 
  (same report engine and such) // need to discuss
* what I missed

-- 
Egor Pasko