Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Received-SPF: pass (herse.apache.org: domain of pavel.ozhdikhin@gmail.com
 designates 64.233.166.182 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=DCFiFtTTxnukSOy1wK1Ojnq6u3Bxi2RTl5XreL/4Jpvl/vLG01kxOFpfbj2902huDahdZtXmEQc34xNOb5uv/RBvJNgSvMHm3+obgUb1kgMK9j5txtPUo5aRB4jYCYvhWjhXKJE1kIEKTZUNXOKgYouPKYSJXLgymOw5aGJYb5w=
Message-ID: <469bff730611160240y2c6176deu62fbfaeb53142811@mail.gmail.com>
Date: Thu, 16 Nov 2006 16:40:06 +0600
From: "Pavel Ozhdikhin" <pavel.ozhdikhin@gmail.com>
To: harmony-dev@incubator.apache.org
Subject: Re: [drlvm][unit] 100% of class library tests pass
In-Reply-To: <8E389A5F2FEABA4CB1DEC35A25CB39CE73C8D6@mssmsx411>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_57272_296514.1163673606401"
References: <8E389A5F2FEABA4CB1DEC35A25CB39CE73C8D6@mssmsx411>

------=_Part_57272_296514.1163673606401
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Sorry to say but it actually does not work until there is no notifications
to the mailing list and no immediate reaction to the regressions.

thanks,
Pavel


On 11/16/06, Fedotov, Alexei A <alexei.a.fedotov@intel.com> wrote:
>
> Pavel, All,
>
> Let me add one "pro" for the second approach: it works already.
> Vladimir's scripts daily upload test results to http://harmonytest.org.
>
> With best regards,
> Alexei Fedotov,
> Intel Java & XML Engineering
>
> >-----Original Message-----
> >From: Tim Ellison [mailto:t.p.ellison@gmail.com]
> >Sent: Thursday, November 16, 2006 12:37 PM
> >To: harmony-dev@incubator.apache.org
> >Subject: Re: [drlvm][unit] 100% of class library tests pass
> >
> >Pavel Ozhdikhin wrote:
> >> We have to evolving systems - classlib and DRLVM. To check commits to
> >> classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise
> >it's
> >> impossible to use DRLVM for pre-commit testing - you never know
> whether
> >> your
> >> test fail because of your patch or due to latest changes in DRLVM.
> >>
> >> I remember the time when DRLVM and Jitrino actively evolved - for
> some
> >time
> >> JIT had to use an older version of DRLVM which could pass all commit
> >> criteria because newer versions suffered from regressions. And
> finally we
> >> came to comon strict commit criteria which prevented regressions in
> both
> >VM
> >> and JIT.
> >>
> >> To avoid regressions using DRLVM in classlib testing I see 3 possible
> >> solutions:
> >>
> >> 1. Use one fixed DRLVM version which can pass 100% HUT test. Update
> this
> >> version from time to time.
> >>    Pros: + Less time to run DRLVM pre-commit tests
> >>              + Classlib does not suffer from regressions in DRLVM
> >>    Cons: - DRLVM will suffer from regressions
> >>               - Classlib can not use the latest DRLVM
> >>               - Need additional efforts to regain stability on DRLVM
> >>                 when we want to update the version for classlib
> testing
> >>
> >> 2. Add HUT to CruiseControl testing on DRLVM and rollback commits
> causing
> >> regressions
> >>    Pros: + Less time to run DRLVM pre-commit tests
> >>              + Classlib can use the latest DRLVM
> >>    Cons: - Classlib can suffer from DRLVM regressions (time lag
> before
> >> rollback)
> >>               - It is not always clear which commit caused a
> regression
> >>               - Rollbacks are costly
> >>
> >> 3. Add HUT to the commit criteria for DRLVM
> >>     Pros: + Classlib always can use the latest DRLVM
> >>               + DRLVM has no regressions regarding to HUT
> >>     Cons: - More time to run DRLVM pre-commit tests (I was told that
> HUT
> >>                   take 25 minutes running in single JVM mode)
> >>
> >> I think that preventing a problem is better than solving it
> afterwards.
> >So,
> >> I personally would choose the 3rd approach, don't mind against the
> second
> >> and dislike the first one. Probably some combination of these is
> >possible.
> >
> >While I appreciate the desire to keep things stable, I think it is
> >unreasonable to ask developers to run the entire test suite each time.
> >As we have seen in the classlib code, running targeted tests before
> >commit and leaving the build machines to run continuous tests ensures
> >that we are productive and are notified of breakages in time to easily
> >back out a patch and re-evaluate.
> >
> >With the amount of machine time we have running harmony tests on
> >different cpu's/os's/compilers/etc we are getting better coverage than
> >any individual could be expected to provide.
> >
> >Which is a long way of saying I think option (2) above is best -- and
> >relies on the bid machines letting us know if things break, and the
> >commitment from all of us to go straight in and fix it.
> >
> >Regards,
> >Tim
> >
> >--
> >
> >Tim Ellison (t.p.ellison@gmail.com)
> >IBM Java technology centre, UK.
>

------=_Part_57272_296514.1163673606401--