Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of aleksey.shipilev@gmail.com
 designates 74.125.46.157 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=DUEezQGob5KL5EwoNxUjfopb+hLtin0aV6ptYpBTIpyeE4bueUh4OH6sHMQ7uqfxmy
         869Qpfzxwc8DrSIW5DqijK9zmV9MD/0rbt4CeiPijUdU4ntAvh5UmqS7AOwXa6ygNVWp
         vwTFF3hqgfdWitsmtTXAWPdtNDbP7R45ys2f4=
Message-ID: <4bebff790812020636w35c1e411p1b7fdc906b4c7c31@mail.gmail.com>
Date: Tue, 2 Dec 2008 17:36:55 +0300
From: "Aleksey Shipilev" <aleksey.shipilev@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [drlvm][jitrino][test] Large contribution of reliability test
 cases for DRLVM+Dacapo
In-Reply-To: <0vqbpvufyrp.fsf@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <4bebff790811231514g3bca2d00ob2ecbfa8753b15db@mail.gmail.com>
	 <0vqtz9nf0mv.fsf@gmail.com>
	 <4bebff790812020048ub64b7bcpc52f2b18fe1929b5@mail.gmail.com>
	 <0vqoczugbnx.fsf@gmail.com>
	 <4bebff790812020423l21aee858uc107fa8c557c3513@mail.gmail.com>
	 <0vqbpvufyrp.fsf@gmail.com>

Hi, Egor!

On Tue, Dec 2, 2008 at 5:16 PM, Egor Pasko <egor.pasko@gmail.com> wrote:
> Aleksey, is there a combined solution, where I push the red button,
> which makes the silver bullet to fire? :)
If the gun's license was granted to Schr=F6dinger... :)

> My concern is: is it effective to look through configurations one by
> one to find issues in this compiler?
Of course not! My original thought was like following: automatize the
check process first, then:
 a. Slide to the next bug.
 b. Reproduce it:
 c1. Is this a bug? Mark it with the assert() or something.
 c2. Is this crash "normal"? Throw some meaningful message.
 d. Re-run all tests and see which fall into new category, mark all
emconfs as faulty within this category.
 e. Exclude new category bugs from the pool and go to (a).

It's more like Sieve of Eratosthenes :)

> However, there is one idea .. why are you classifying the
> configurations based only by end result status? clusters are obviously
> too big. I would also take the configurations as a parameter for
> clustering failures. Yes, you'll need a fair amount of machine
> learning efforts to cluster them. But that may pay off really
> well.
Yeah! Glad we think in one direction -- try to cluster the bugs. I was
solving the similar issue in my school project with Kohonen's maps,
hopefully I'll reproduce the project some day for this occasion. We
can also look into Matlab ;) Anyway, this sounds like a good student
project "just-for-fun".

Thanks,
Aleksey.